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Description 

METHOD AND APPARATUS FOR RECORDING AND REPRODUCING SENSOR DATA 

5 Technical Field 

The present invention relates to methods and apparatus for 
recording and reproducing signals and data and, more particularly, 
to methods and apparatus for recording and reproducing sensor data 
and video signals. 

10 

Background of the Invention 

Motor vehicles (including personal automobiles, transit buses 
and transit trains) are an integral part of modern societies. 
Beside providing the facility of transporting goods in the market 

15 place, they also transport millions of people on a daily basis and 
are used to provide general services, such as public safety 
services. The burgeoning use of motor vehicles has given rise to a 
complex and expensive liability system concerning vehicular 
transportation. In nearly all states of the United States, for 

20 example, automobile insurance is required. Accordingly, much 
expensive litigation ensues in connection with motor vehicle 
accidents and other incidents. 

The massive system of policies and claims that result from 
the insurance coverage and related litigation are subject to 

25 fraudulent abuse. Current estimates indicate that this phenomenon 
is on the rise, causing increased cost to the system, even in 
locales where accident rates are diminishing. In addition to 
fraud, the resolution of claims relating to motor vehicles is an 
inefficient time-consuming process that experiences additional 

30 costs by relying heavily on a tort-based legal process. As one 
measure of the increased burden that this phenomenon puts on 
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society, legal costs concerning automotive and transit bus injury 
claims is increasing significantly. 

One problem related to these increased costs is the 
difficulty in providing basic documented facts concerning motor 
5 vehicle (e.g., automobile and transit bus) incidents. Beside the 
abuse of the process allowed by parties bringing suit in the hopes 
of an out-of-court settlement, costs are often significantly 
greater when a motor vehicle-related suit does end up in court. To 
combat these problems of documentation and out -of -control costs, 

10 many insurance companies and states in the United States are 

considering implementation of mandatory vehicle inspections and 
special investigation personnel units. However, these approaches 
are expensive, too, and this expense adds to the overall costs of 
the transportation system. 

is One approach to helping curtail these costs is to provide 

better data concerning motor vehicle incidents. This will help to 
curtail fraudulent claims. While other vehicle-borne systems have 
been proposed to provide information concerning automotive and 
transit bus accidents, these systems are themselves subject to 

20 fraudulent manipulation. As one result, courts of law are not 

likely to give much evidential weight to such information systems 
unless they are secure. 

Accordingly, it is desirable to have a vehicle-borne data 
collection system which is capable of securely recording data 

25 concerning accidents. Of course, such systems will have other 
applications, such as other means of transportation and other 
situations demanding secure preservation of documentary data. 
Other applications concerning automotive transportation include 
taxis, limousines and rental vehicles. Depending upon the 

30 circumstances, the documentary data can include video data (and 

accompanying audio data) concerning occurrences exterior to and/or 
within the automotive transportation unit, as well as data related 
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to operation of the unit such as dynamic variables (for example, 
position, velocity and acceleration) and data provided by other 
analog and digital sensors. 

Examples of alternative transportation systems (aside from 
5 automotive transportation units) include transit buses and 

trolleys, subway and rail cars, air- and water-borne craft and 
elevators. Other, non- transportation systems include stationary 
systems, such as bank, other building, and location (such as ATM 
location) security systems. 

10 In another area of applications, the increasingly litigious 

societal environment has led to an increased need for a 
transportable way to document the circumstances of other (i.e., 
other than motor vehicle) incidents. Examples of such incidents 
include incidents that should be documented by police and other 

is public safety officers as well as non -governmental persons such as 
insurance and security agents and other technicians. Such 
documentation is typically accomplished through the use of 
recorded still and moving images as well as outputs of other 
analog and digital sensors . As electronics is continues to have 

20 even further impact on photographic and videographic processes, it 
is natural that photographic and videographic equipment be used 
for these documentary purposes. As described above, there is a 
definite need to document and validate the images and sounds taken 
when doing the documentation. 

25 Regardless of whether the video and/or sensor data are 

collected by a system that has components in potentially insecure 
locations, an additional level of data security can be provided by 
transmitting (or transporting) the data to a remote location. 
Aside from the additional security provided, transmitting the data 

30 to a remote location allows the data to be processed as it is 
being collected. In some circumstances, this processing can be 
used to assure system managers that the video sources and/or 
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sensors are operating properly. In other circumstances, remote 
processing of data can be used to determine whether the data 
should be collected in some other manner or form, such as by 
repositioning the video sources (possibly by means of control 
5 signals transmitted through a reverse transmission link) . 

Summary of the Invention 

According to one aspect, the invention is a method for 
recording sensor data with a resultant video signal to protect the 

10 sensor data and the resultant video signal. The method includes 
the steps of a) selecting one or more sensors that produce the 
sensor data and b) transforming the signals from the one or more 
sensors to digital signals . The method further includes the steps 
of c) selecting one or more video signal sources from a set of two 

is or more video signal sources producing respective video signals 
and d) producing the resultant video signal from the respective 
video signals produced by the selected one or more video signal 
sources . 

In addition, the method includes the steps of e) encrypting at 
20 least one of the digital sensor signal and the resultant video 
signal in accordance with a digital encryption procedure to 
produce a respective digital output sensor signal and a respective 
digital output video signal, f) combining the respective digital 
output sensor signal and the respective digital output video 
25 signal to produce a digital output signal and g> recording the 
digital output signal on a recording medium. Step a) includes 
digitizing analog signals produced by some of the sensors and 
combining the digitized analog signals with digital signals 
produced by the remaining sensors to produce a digital sensor 
30 signal. 

According to a further aspect, the invention is an apparatus 
for recording sensor data with a resultant video signal to protect 
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the sensor data and the resultant video signal . The apparatus 
comprises a selection circuit to select one or more sensors that 
produce the sensor data and a digitization circuit to transform 
the signals from the one or more sensors to digital signals. The 
5 digitization circuit digitizes analog signals produced by some of 
the sensors and combines the digitized analog signals with digital 
signals produced by the remaining sensors to produce a digital 
sensor signal. The invention further includes a selection circuit 
to select one or more video signal sources from a set of two or 

10 more video signal sources producing respective video signals and a 
video production circuit to produce the resultant video signal 
from the respective video signals produced by the selected one or 
more video signal sources. In addition the invention includes an 
encryption circuit to encrypt at least one of the digital sensor 

is signal and the resultant video signal in accordance with a digital 
encryption procedure and to produce a respective digital output 
sensor signal and a respective digital output video signal, a 
combination circuit to combine the respective digital output 
sensor signal and the respective digital output video signal to 

20 produce a digital output signal and a recorder to record the 
digital output signal on a recording medium. 

According to one further aspect, the invention is a method 
for recording sensor data with a video signal to protect the 
sensor data. The sensor data is contained in signals from one or 

25 more sensors. The method comprises the steps of a) encrypting the 
signals from the one or more sensors, b) combining the video 
signal and the encrypted signals to produce an output signal, and 
c) recording the output signal on a recording medium. 

According to another further aspect, the invention is a 

30 method for retrieving sensor data originally produced by one or 

more sensors. The sensor data is encrypted with a video signal to 
produce a recorded signal that protects the sensor data and being 
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contained in signals from one or more sensors. The method 
comprises the steps of a) playing back the recorded signal to 
produce a played-back signal, b) decrypting the played-back signal 
to produce a video signal and one or more sensor signals 
5 containing data originally produced by one or more sensors, and c) 
processing the one or more sensor signals to produce the sensor 
data . 

According to still another further aspect, the invention is a 
method for recording sensor data with a video signal to protect 

o the sensor data. The sensor data is contained in signals from one 
or more sensors. The method comprises the steps of a) compressing 
the video signal to produce a compressed video signal, b) 
combining the compressed video signal and the signals from the one 
or more sensors to produce a digital signal, c) encrypting the 

5 digital signal to produce an output signal, and d) recording the 
output signal on a recording medium. 

According to yet another further aspect, the invention is a 
method for retrieving sensor data originally produced by one or 
more sensors. The sensor data is encrypted with a video signal to 

o produce a recorded signal that protects the sensor data and being 
contained in signals from one or more sensors, at least one of the 
sensors producing an analog signal. The method comprises the steps 
of a) playing back the recorded signals to produce a played-back 
signal, b) decrypting the played-back signal to produce a video 

5 signals and one or more sensor signals containing data originally 
produced by one or more sensors, c) determining whether any of the 
one or more sensor signals are digital signals, and d) processing 
the one or more sensor signals to produce the sensor data, 
including converting the one or more sensor signals to produce a 

o corresponding analog signal. 

According to a still further aspect, the invention is a 
method for recording sensor data with a video signal to protect 
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the sensor data. The sensor data is contained in signals from one 
or more sensors. The method comprises the steps of a) determining 
which of the signals from the one or more sensors are analog 
sensor signals and which of the signals from the one or more 
5 sensors are digital sensor signals, b) converting the data in the 
analog sensor signals to one or more first digital signals, c) 
encrypting the digital sensor signals and the one or more first 
digital signals to produce second digital signals, d) converting 
the second digital signals to first analog signals, and e) 

10 recording the video signal and the first analog signals on a 
recording medium. 

According to yet another aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 

is or more sensors. The apparatus comprises an encryption circuit, a 
signal processing circuit and a recorder. The encryption circuit 
encrypts the signals from the one or more sensors. The signal 
processing circuit combines the video signal and the encrypted 
signals to produce an output signal, and the recorder records the 

20 output signal on a recording medium. 

According to yet another aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 
or more sensors. The apparatus comprises a compression circuit, a 

25 signal processing circuit, an encryption circuit and a recorder. 
The compression circuit compresses the video signal. The signal 
processing circuit combines the compressed video signal and the 
signals from the one or more sensors and produces a digital signal 
therefrom. The encryption circuit encrypts the digital signal and 

30 produces an output signal therefrom. The recorder records the 
output signal on a recording medium. 

-7- 

BNSDOCID: <WO 9723075A1J_> 



r 



WO 97/23075 PCT/US96/20912 

According to an even further aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 
or more sensors. The apparatus comprises a distinction circuit, a 
5 first conversion circuit, an encryption circuit, a second 
conversion circuit, and a recorder. The distinction circuit 
determines which of the signals from the one or more sensors are 
analog sensor signals and which of the signals from the one or 
more sensors are digital sensor signals. The first conversion 

10 circuit converts the data in the analog sensor signals to one or 
more first digital signals. The encryption circuit encrypts the 
digital sensor signals and the one or more first digital signals 
to produce second digital signals. The second conversion circuit 
convert the second digital signals to first analog signals, and 

15 the recorder records the video signal and the first analog signals 
on a recording medium. 

According to even still another aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 

20 or more sensors. The apparatus comprises means for encrypting, 
means for combining and means for recording. The means for 
encrypting encrypts the signals from the one or more sensors . The 
means for combining combines the video signal and the encrypted 
signals to produce an output signal . The means for recording 

25 records the output signal on a recording medium.* 

According to another aspect, the invention is a method for 
retrieving sensor data originally produced by one or more sensors, 
the sensor data being encrypted with a video signal to produce a 
recorded signal that protects the sensor data and being contained 

30 in signals from one or more sensors, at least one of the sensors 
producing an analog signal. The method comprises the steps of a) 
playing back the recorded signals to produce a played-back signal, 
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b) decrypting the played-back signal to produce a video signals 
and one or more sensor signals containing data originally produced 
by one or more sensors, c) determining whether any of the one or 
more sensor signals are digital signals, and d) processing the one 
5 or more sensor signals to produce the sensor data, including 
converting the one or more sensor signals to produce a 
corresponding analog signal . 

According to and even still further aspect, the invention is 
an apparatus for recording sensor data with a video signal to 

10 protect the sensor data. The sensor data is contained in signals 
from one or more sensors. The apparatus comprises means for 
compress, means for combining, means for encrypting and means for 
recording. The means for compressing compresses the video signal 
to produce a compressed video signal . The means for combining 

15 combines the compressed video signal and the signals from the one 
or more sensors to produce a digital signal. The means for 
encrypting encrypts the digital signal to produce an output 
signal, and the means for recording records the output signal on a 
recording medium. 

20 In still yet another aspect, the invention is an apparatus 

for recording sensor data with a video signal to protect the 
sensor data. The sensor data is contained in signals from one or 
more sensors. The apparatus comprises means for determining, first 
and second means for converting, means for encrypting, and means 

25 for recording. The means for determining determines which of the 

signals from the one or more sensors are analog sensor signals and 
which of the signals from the one or more sensors are digital 
sensor signals. The first means for converting converts the data 
in the analog sensor signals to one or more first digital signals. 

30 The means for encrypting encrypts the digital sensor signals and 
the one or more first digital signals to produce second digital 
signals . The second means for converting converts the second 
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digital signals to first analog signals, and the means for 
recording records the video signal and the first analog signals on 
a recording medium. 

Brief Description of the Drawings 

Figure 1 is a schematic diagram of a first preferred 
embodiment of an apparatus of the invention. 

Figure 2 is a schematic diagram of the structure of a data 
file, as stored by the preferred embodiments of the invention. 

Figure 3 is a schematic diagram of a second preferred 
embodiment of an apparatus of the invention. 

Figure 4 is a flow chart of the program running within the 
preferred embodiments of the invention. 

Figure 5 is a representation of a video image, illustrating a 
method for controlling and assuring the integrity of the image 
upon processing, storage and playback. 

Figure 6 is a block diagram of an apparatus for playback of 
the data stored by the data recorder of the present invention. 

Figure 7 is a flow chart of the method of playing back the 
data stored by the data recorder of the present invention. 

Detailed Description of the Invention 

Figure 1 is a schematic diagram of a preferred embodiment of 
an apparatus of the invention. The sensor data recorder 10 can be 
adapted for placement at or near the location where the sensors 
are located. In the following detailed description, it will be 
assumed that the data recorder 10 is located in an automobile (not 
shown) . Within the automobile, the data recorder 10 can be placed 
in a passenger compartment or, for greater security, can be placed 
in a more secure place such as a trunk. Of course, those skilled 
in the art would provide appropriate protection for the data 
recorder 10, depending upon the particular task to which the data 
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recorder 10 is applied. For example, a data recorder 10 that is to 
be installed in an automobile should probably be shielded from 
accelerations due to impacts, as well as temperature extremes and 
high humidity. Other applications will obviously have their own 
5 particular environmental factors to be considered in the 
installation . 

In the preferred embodiment, the data recorder 10 includes a 
number of sensors, such as ignition switch 11, brake sensor 12, 
accelerator sensor 14, steering column sensor 16, wheel rotation 

10 sensor 18, turn signal sensor 20 and ultrasound detection sensor 
21. The data recorder 10 also includes a source of video 
information such as a forward- facing charge -coupled device (CCD) 
microcamera 22 and a recorder 24 . The data recorder 10 may also 
include one or more optionally additional CCD microcameras , such 

:5 as CCD microcamera 25 that can be pointed rearward. Of course, in 
non- automotive applications the sensors 11, 12, 14, 16, 18, 20 and 
21 can be augmented or substituted by other sensors that are 
dependent upon the application of the data recorder 10 . For 
example, the sensors can include one or more microphones 13 and a 

20 sensor 15 (such as a global position satellite, GPS, sensor) 
capable of measuring dynamic variables such as position and 
velocity to sufficient accuracy for the task of the data recorder 
10. The sensors can also include one or more simple switches 17 
(possibly contained in a keyboard) for inputs by a human 

25 observer /user . 

The CCD microcamera 22 can be a conventional camera mounted 
on a universal mount. Appropriate specifications are that the CCD 
microcamera 22 should be operable in an operating temperature 
range from -10 Celsius to +60 Celsius and to a maximum relative 

30 humidity of 95%. Further, the CCD microcamera 22 should have 

automatic shutter sensitivity ranging from 1/60 to 1/3 2000 second 
and a resolution of at least 380 by 3 80 picture element (pixel) 
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rows and columns. In addition, the CCD microcamera 22 should have 
an illumination sensitivity of 1 lux or less. 

In some embodiments, it may be desirable to place the CCD 
microcamera 22 on a pan/tilt mount in a weatherproof enclosure. 
5 The CCD microcamera 22 could then be mounted outside of a vehicle 
and equipped with radio, infrared or ultrasound beacon- following 
sensors. Implementation of such sensors are well within the skills 
of those of the relevant electronic arts. Such beacon- following 
systems can be used to follow and provide video images of a person 

10 carrying an appropriate beacon source. An important example is a 
microcamera adapted to follow public safety personnel who are 
wearing a beacon and whose motions and surroundings can then be 
automatically recorded in video form. Further, if a radio or 
infrared beacon is used, such a system could also provide an audio 

is signal from the vicinity of the person wearing the beacon. The 
audio signal could carry the voice of the person wearing the 
beacon, and could be compressed, encrypted and recorded with the 
data from the other sensors, as will be described below. 
Alternatively, the pan/tilt mount can be controlled by a motor 

20 drive that receives control signals from external sources, as will 
be described in further detail subsequently. 

The recorder 24 can be a conventional JVC time lapse cassette 
recorder, although other recorders (as described subsequently) 
could also be used. 

25 The -sensors 11, 12, 13, 14, 15, 16, 17, 18/ 20 and 21, and 

the CCD microcamera 22 (and CCD microcamera 25, if included) are 
connected to the recorder 24. Also, a display 23 can be connected 
to the recorder 24 (for example, to the sensor I/O circuitry 50) 
for use by a human user /operator . 

30 The various sensors 11, 12, 13, 14, 15, 16, 17, 18, 20 and 21 

may include a local transducer at the location of the variable 
being sensed and a remote transducer placed more closely to the 
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recorder 24 than the location of the variable being sensed. For 
example, the wheel rotation sensor 18 may include a local 
transducer that transforms some of the rotational energy into 
electrical energy and a remote transducer that converts the 
5 electrical energy into an electromagnetic signal that can be 
recorded by the recorder 24 . The local transducer of the wheel 
rotation sensor 18 may be a segmented ferrous-based cog wheel that 
rotates with the wheel and the remote transducer can be a 
reluctance-sensing transducer that transforms the detected pulses 

10 in the magnetic field to an electrical signal that is processed 
for optimal use by the data recorder 10 . 

Similarly, the ultrasound detection sensor 21 can have 
forward- or side-located components that produce signals that are 
further processed at a remote location. In one embodiment, the 

15 ultrasound detection sensor 21 can include a plurality of 

ultrasonic transducers mounted at the front and around the sides 
of the vehicle. From those locations, transducers produce and/or 
receive ultrasonic (or electromagnetic) signals to sense or 
determine the immediate surroundings of the vehicle and produce 

20 signals that can be processed to produce an alarm signal that will 
activate the inventive data recorder 10 to record the proximity of 
other nearby vehicles. Depending upon the particular signal 
processing used, in one embodiment such an ultrasound detection 
sensor 21 can be used to monitor the driving habits or the 

25 vehicle's operator(s) . 

The recorder 24 shown in Figure 1 is a hardware-based 
implementation of an embodiment of the invention. In general, the 
recorder 24 includes a mass storage device 30, a bus 32, a signal 
processing circuit 34, a signal conversion circuit 36 and a sensor 

30 input/output (I/O) circuit 50. The mass storage device 30, the 
signal processing circuit 34, and the signal conversion circuit 
36, and the sensors I/O circuit 50 are all connected to the bus 
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32. The mass storage device 30 can take several forms. In one 
preferred embodiment the mass storage device 30 is a magnetic tape 
recorder (specified above) that records on a magnetic tape 
included in a removable cassette. In other embodiments, the mass 
5 storage device 30 can be a solid-state flash memory card or other 
removable memory mediums that store data. Examples are 2.5 and 3.5 
inch removable disk drives capable of storing at least 120 
megabytes of data. 

The signal processing circuit 34 includes a central 

10 processing unit (CPU) 40, a random access memory (RAM) 42 and a 
clock 44 that are connected together in conventional fashion. 
These connected components allow the CPU 40 to process sensor and 
other signals received from the bus 32 in accordance with clock 
signals produced by the clock 44 and to record the results of the 

is signal processing operations in the RAM 42. The actual signal 
processing performed by the CPU 40 is dependent upon a program 
which is installed in the CPU 40 when the data recorder 10 is 
first turned on. Such programming is conventional to those skilled 
in the art of computer programming. Two aspects of the necessary 

20 programming are worthy of specific note, however. The first is 

that the CPU 40 is programmed to convert digital signals from one 
format to another. The second is that the CPU 40 is also 
programmed to encrypt the information in the signals. The actual 
encryption method can be any one of many available possibilities. 

25 The preferred form, however, is a public key encryption method 
chosen from the many known public key encryption methods. 

The signal conversion circuit 36 is used to convert the video 
signal from the CCD microcamera (s) 22 (and 25) . At a minimum, the 
signal conversion circuit 3 6 must convert the analog video signal 

30 to digital data. Further, the signal conversion circuit 3 6 can 

control the sequencing of the video signals if there is more than 
one microcamera. The signal conversion circuit 36 can include a 
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multiplexer that controls which CCD microcamera is being recorded 
at a given time. As will be described below, the rate at which the 
video signals are switched can be made a function of the state of 
alarm of the recorder 10 . Accordingly, the signal conversion 
5 circuit 3 6 can be made to respond to alarm signals produced by the 
signal processing circuit 34. Alternatively, the signal conversion 
circuit 36 can take video signals from all of the CCD microcameras 
in operation and produce a concurrently composite video signal, 
such as a split screen video signal. The signal conversion circuit 

10 36 can also receive date and time signals from the signal 

processing circuit 34 and produce video representations of those 
date and time signals in the analog video signal before digitizing 
the video signal. 

In some versions of the preferred embodiment of Figure l, the 

15 signal processing circuit 34 can advantageously be responsive to 
the outputs of one or more of the sensors 11, 12, 13, 14, 15, 16, 
17, 18, 19, or 20. As an important example, if the recorder 10 is 
installed in a transit bus, it may be advantageous to have four 
microcameras (such as microcameras 22 and 26) connected, all 

20 directed toward the interior of the transit bus. Two of the 

microcameras can be located at the front and rear ends of the bus 
and the other two microcameras can be located to monitor the 
transit bus' entry/exit doors. The signal processing circuit 34 
can be caused to switch from the fore- and aftward pointing 

25 microcameras to the door-monitoring microcameras as a function of 
the position of switches 17 which can be connected to the 
entry/exit doors in such a way that determines whether the doors 
are closed. When the switches 17 indicate that the entry /exit 
doors are closed, the signal processing circuit 34 causes the 

30 video signals from the fore-and aftward pointing microcameras to 
be used, while the door monitoring microcameras are used when the 
doors are open. 

-15- 

BNSDOCID: <WO 972307 5A1 J_> 



WO 97/23075 



PCT/US9<i/20912 



As a further embellishment of the transit bus system just 
described, the microcameras whose signals are being recorded at 
any time can be a function of time or location (via the GFS sensor 
15) . Without limitation, such a system can be used to monitor 
5 events exterior to the transit bus when the bus is passing through 
a high crime area. Also such a system can be used to monitor 
specific areas with the microcameras in response to an emergency 
event signified by actuation of a switch 17 (by the transit bus 
operator, for example) or by actuation of the accelerator sensor 

10 14. It is possible to use the accelerator sensor 14 to sense the 
occurrence of a collision when the acceleration levels sensed 
exceed a predetermined threshold. 

Further, when desired, it is possible designate special 
events which will be preserved as inviolate data to be recorded by 

15 the recorder 24. As an example, it is possible that the transit 
bus operator may want to record events that occur within and/or 
exterior to the transit bus and to preserve those events without 
fear that they will subsequently be written over inadvertently. 
Typically, such a recording will include a fixed predetermined 

20 period of time, including periods of time before and after the 

switch 17 is actuated. In one system, the first actuation of the 
switch 17 can specify that a recordable event has occurred and the 
next subsequent actuation of the switch 17 can specify the end of 
the event. The recorder 24 can then adjust the portion of the 

25 video and sensor signals that it preserves. • 

In addition, the signal conversion circuit 3 6 compresses the 
highly redundant video signals from the CCD microcamera 21 and 
produces therefrom a compressed digital signal that it trc.nsfers 
to the bus 32. The compression operation is performed by hardware 

30 in the signal conversion circuit 36. This hardware conventionally 
takes the form of an application-specific integrated circuit 
(ASIC) that performs JPEG compression. Alternatively, and in some 
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applications, it may be preferable to perform compression of the 
video data through other hardware means, such as an ASIC that 
performs MPEG or other well-known compression techniques. 
Specifically, the compression portion of the conversion circuit 36 
compresses data received from the CCD microcamera 22, as well as 
any of the optional additional CCD microcameras 25 that may be in 
use in the specific preferred embodiment that is suitable for a 
particular application. It is also possible to perform a 
simultaneous compression and encryption operation at this stage of 
the signal conversion process. 

In operation, the recorder 24 receives signals from the 
sensors 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 and 21, through the 
sensor I/O circuit 50 and the bus 32. The recorder 24 also 
receives signals from the CCD microcamera 22 (and any possible 
optional additional microcameras 25) through the signal conversion 
circuit 36, The signal processing circuit 34 receives the sensor 
and digital video signals through the bus 32, processes the 
signals, and sends the resulting processed digital signals to the 
mass storage 30 for recording. 

Since the data recorded in the mass storage is encrypted, its 
security during playback can be given a very high degree of 
assurance. If a party wished to alter the data recorded in the 
mass storage, the encryption system applied would have to be 
defeated and the recorded data recorded over with counterfeit data 
that make contextual sense. In general, it would be very difficult 
to do so. It would be impossible, even given the massive amount of 
computing power available today, to counterfeit the data that can 
be stored by the present invention if a time constraint is 
applied, even a time constraint measured in years. 

In some applications, it may be desirable to omit the signal 
conversion circuit 36 and transfer the data from the CCD 
microcamera 22 directly to the bus 32. In other applications, it 
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may be desirable to omit the encryption of the digital data 
received by the signal processing circuit 34. 

As applied to one particular desired specific embodiment of 
the invention, the signals from the CCD microcamera 22 can be 

5 compressed by the compression circuit (such as a JPEG circuit) . 
Then the compressed data, along with the sensor data can b-2 mixed 
with clock data before encryption, so that the encrypted data 
includes contemporaneous recordings of the date and time ( zlock 
signal) , the sensor positions (sensor data) and the visual scene 

o (video data) . If desired, the date and time information can be 

inserted, into the visual scene (through conventional circuitry and 
processing, not shown) . 

It may be desirable to retain remote copies of the data 
collected by the recorder 24. In some versions of the preferred 

s embodiment, it may be the primary function of the recorder 24 to 
transmit the data to a remote location, while . retaining a copy of 
the data transmitted for backup or archive purposes. In addition 
to storing and/or processing the data, such a system can provide 
control signals to the recorder 10 to alter the response and/or 

o configuration of the recorder 10. To accomplish this, the recorder 
10 can also include a controller 200, a transceiver 202, and an 
antenna 206 that communicates with a remote transceiving 
system/computer 208. The controller 200 is connected to the bus 
32, the transceiver 204 is connected to the controller 200, and 

5 the antenna 206 is connected to the transceiver » 204 . When data 
collected, compressed and encrypted by the recorder 10 is to be 
transmitted to the remote location, the data are prepared by the 
recorder 10 and then sent to the controller 200 over the fcus 32. 
the signals representing the data to be transmitted are augmented 

10 by appropriate control signals, to which the controller 2C0 

responds. The controller 200 then controls the transceiver 204 and 
causes it to transmit the data to the remote location through the 

-18- 

9723075A1„I_> 



WO 97/23075 



PCT/US96/20912 



antenna 206. Of course, it would be well-known to those skilled in 
the art to transmit to the remote location via electromagnetic 
energy . It would also be well-known * to use ultrasonic radiation 
to transmit signals to a remote location. 
5 After the data are received at the remote location, the 

remote cransceiving system/computer 2 08 can be configured to 
record and process the data. Depending upon the results of the 
data processing, it may be desirable to transmit control signals 
from the remote transceiving system/computer 208 to the recorder 

10 10. To accomplish this, the control signals are transmitted to the 
antenna 206 by the remote transceiving system/computer 208, the 
received signals being demodulated by the transceiver 204 and 
passed on to the controller 200 for further processing. Any data 
included with the data signals can be separated by the controller 

15 200, which then responds to t'he control signals and causes the 

data to be transmitted to the appropriate portion of the recorder 
10. As a simple example, the remote transceiving system/computer 
208 may produce control signals to activate a sensor or to cause 
one or the other of the microcameras 22 and 25 to be repointed to 

20 collect video data from a desired direction. 

Figure 2 is a schematic diagram of the structure of a data 
file, as stored by the preferred embodiment of the invention. The 
data file 60 is represented as a rectangular area containing 
digital data which are stored in a memory such as the RAM 42 of 

25 the signal processing circuit 34 in Figure 1. However, the data 
have a format which is useful when reading and playing back the 
data. Specifically, the data file contains header data 62, video 
data 64 and sensor data 66, in that order. The header data 62 
includes an indication of the file format that is used in the 

3 0 storage of the data in the file, the version of the recorder on 
which the data were recorded, a customer identification number, 
and additional portions reserved for future use. The video data 64 
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is 8 -bit gray scale data, representing 256 different levels of 
gray. Alternatively, the video data 64 can represent a color video 
image. The video data 64 can be postprocessed, after the analog 
video signals have been digitized, by the insertion of a special 
digital data portion 68 that contains documentary data. Most often 
the documentary data will be the date and time at which this images 
in the video data were imaged by the microcamera (s ) 22 and 25. 
However, the documentary data could be expanded to include 
digitized representations of the outputs of some or all of the 
sensors 11, 12, 14, 16, 18, 20 and 21. 

While this procedure may document the video data 64, it can 
be useful to provide a first level of security to the video data 
by reserving specific pixels represented by the video data 64 
which are used by the fonts of the characters used to represent 
the documentary data in the digital data portion 68. The f^nt is 
chosen so that it is very difficult to reproduce if an attempt is 
made to modify the video data 64, but is very easy to verify as 
valid if no attempt to modify the video data 64 has been made. 

The sensor data 66 includes digital representations cf the 
value of the variable (s) for each of the sensors 11, 12, 14, 16, 
18, 20 and 21. Typically, it is desirable to encode the digital 
sensor data 66 to discourage tampering. However, the primary 
security for the sensor and other data in the file format is 
provided by encryption that is applied after the data in the file 
are compressed. • 

There are further procedures to heighten the security of the 
data, particularly the video data 64 stored in the file format 60. 
One approach is to encode the gray scale (or color) information in 
the file format 60 so that consecutive digital values do not 
represent consecutive grays or colors. In this way, the data are 
represented in a false gray (or color) fashion that will 
discourage tampering . 
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Figure 3 is a schematic diagram of a second preferred 
embodiment of an apparatus of the invention. The recorder 24 shown 
in Figure 3 is a software-based implementation of an embodiment of 
the invention. In this schematic diagram, components which serve 
5 the same purpose as those described in Figure 1 are given the same 
reference numerals as their counterparts in Figure 1. 

In Figure 1, ignition switch sensor 11 may take the form of a 
conventional ignition switch that can be accessed so that it is 
possible to determine whether the ignition switch is turned on. 

10 Turning on the ignition is generally a good signal that the 

vehicle is about to be operated and may, in some system, signify 
that the data recording process should begin. . In addition, the 
brake sensor 12 may take the form of a brake pressure sensor, the 
accelerator sensor 14 may take the form of a throttle position 

15 sensor, and the steering column sensor 16 may take the form of a 
wheel position sensor. 

The sensors can also include a speed input sensor 23 
including a toothed wheel that rotates as the vehicle moves and 
produces pulsed signals in accordance with the rotational speed of 

20 the wheel . Another optional sensor is a Global Positioning 

Satellite receiver 52 which receives positional signals through 
its roof-mounted antenna 54 and produces sensor signals that are 
used by the sensor data recorder 10 to indicate the global 
location of the vehicle in which it is installed (within 30 

25 meters) . A further optional sensor is a door open sensor 56. The 
door open sensor 56 can be useful in identifying situations in 
which it is desirable to start recording data, even if the vehicle 
is not started. It can also be useful in identifying situations in 
which a public safety officer has left the vehicle, signifying 

30 that a beacon-following system controlling the pointing direction 
of the CCD microcamera (s) 22 and 25 should be activated. 
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The mass storage device 30 of the recorder 24 can be a 
removable hard disk having 500 megabytes of storage capacity. The 
recording time for the mass storage device is dependent upon the 
sampling rate of the signals which are to be stored. 
5 The signal conversion circuit 3 6 is connected to one or both 

of the CCD microcameras 22 and 25 and includes flash converter 
circuitry to quickly transform the analog video signals from the 
microcameras to digital form. In this form, and in coordination 
with the signals produced by a system timing circuit, a frame 
10 buffer circuit produces framed digitized video data that are 
transferred to the bus 32 for transport to other hardware 
components of the data recorder 10, such as the signal processing 
circuit 34 . 

The signal processing circuit 34 includes a CPU, such as an 

15 Intel 803 86 microprocessor, which is connected to a dynamic RAM 
and to a real-time clock in accordance with the good engineering 
practices well-known to those skilled in the electronics hardware 
arts. The data produced by the microprocessor are transmitted to 
the bus 32, which transfers them to the appropriate hardware 

20 component of the sensor data recorder 10, such as the mass: storage 
device 30. In specific application, the microprocessor of the 
signal processing circuit 34 receives the data containing header, 
video and sensor data according to the file format 60 (see Figure 
2) . As the data are received, the microprocessor applies a 

25 compression algorithm, such as the LWZ algorithm, to the received 
data and stores the data in compressed form in the dynamic: RAM. 
The microprocessor then applies an encryption algorithm to the 
stored data and transfers the resulting encrypted and compressed 
data to the mass storage device 30 through the bus 32. In the mass 

30 storage device 30, the data are stored in a hard disk that; is 

removable from the mass storage device 30. The removable hard disk 
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containing the encrypted compressed data can be stored in a 
convenient location for later retrieval and replay of the data. 

The power supply 70 of the data recorder 10 shown in Figure 3 
is an integral part of the data recorder. The power supply 70 is 
5 normally connected to the vehicle's battery. However, to protect 
the data recorder 10 from unexpected disruption and/or intended 
sabotage to the connection between the power supply 70 and the 
battery, the power supply 7 0 includes a backup battery 72 . The 
backup battery^ 7 0 will supply power to the data recorder 10 in the 

io event that the connection between the power supply 70 and the 

vehicle's battery is broken. In addition, the power supply 70 can 
be used to extend the time that the data recorder 10 can record 
data for a period of time after the automobile's ignition switch 
has been turned off . 

is Figure 4 is a flow chart of the program running within the 

preferred embodiment of the invention shown in Figure 1. The 
program is simultaneously accomplishing two tasks . One task is to 
receive and process data from the sensors. The other task is to 
receive and process data from the video camera (s) 22 (25) and to 

20 combine the video data with the sensor data. The combined data is 
then compressed, encrypted and stored to disk for later retrieval 
and replay in accordance with methods known in the prior art . The 
replay methods comprise the inverse steps to those described below 
in carrying out the encoding, encryption, compression and 

2 5 recording steps. 

In accordance with signals produced by the system clock 
(block 100) , the video data task starts (block 102) and digitizes 
a frame of video data produced by one or both of the microcameras 
(block 104) . The system clock signals control when the readings 

30 from the sensors and the video camera (s) are taken. Typically, the 
sensor signals are sampled four times per second and the video 
camera data are sampled once per minute. However, the data can be 
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sampled at different controllable rates as described below. The 
digitized video data is transferred to the dynamic RAM (block 106) 
and interspersed with time/date signals produced by the system 
clock, the time/date signals characterizing the time and date at 
5 which the signals are produced (block 108) . The sensor data are 

also appended to the stored video and time/date signal data (block 
110) in the file format described above, compressed (block 112) , 
encrypted (block 114) , and stored to disk (block 116) for Later 
retrieval and replay. Then this task of the computer program 

10 awaits a further signal from the system clock, at which tine it 
will begin again at block 102. 

In the other task, the program reads the sensors when the 
appropriate signal is produced by the system clock (block MB) . 
The sensor readings are taken frequently, allowing them to be 

15 processed digitally and/or statistically before they are made 
available as sensor data by the step represented by block :.18. 
This program task then compares the sensor data to trip poi.nts or 
thresholds (block 120) and determines whether an alarm should be 
produced (block 122) . In the alarm condition, when the CCD 

20 microcameras are normally inactive, they are activated to ::ecord 

the events that caused, and the sensor data that resulted ::rom the 
alarm condition. However, if the CCD video microcameras arts 
normally active, their signals may be sampled more frequently in 
the case of an alarm to increase the chance that the event 

25 prompting the alarm condition will be completely, recorded. 

Examples of such trip points are a particular vehicle speed or a 
particular amount of steering wheel motion. In the first example, 
the step of block 12 0 can detect when the vehicle speed exceeds 
the trip point, and the step of block 122 can determine th.it an 

30 alarm should be produced. In the second example, the data recorder 
analyzes the magnitude and frequency of the steering motions to 
establish when the trip point has been exceeded. 
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If an alarm is to be produced, the alarm signal is processed 
(block 124), causing adjustment in the sampling rate, an alarm 
timeout and reset condition are established (block 126) and the 
program returns to block 118 to read the sensors again. To save 
5 storage space in the mass storage device 30, the alarm timeout is 
established to allow the system to collect possibly relevant data 
without requiring an undesirably large amount of the recording 
medium. The duration of the alarm timeout is a function of the 
reason for the«alarm. For example, if the alarm is caused by the 

10 vehicle's speed exceeding a desired threshold, the data recorder 
10 should be activated only as long as the speed is above the 
desired threshold. On the other hand, if the alarm is caused by 
erratic steering movements, the alarm condition should last beyond 
the time when the erratic steering movements cease. 

15 If the alarm is not to be produced, the sensor data are 

stored in an array in memory (block 128) for use by the append 
sensor data step (represented by block 110) of the other task of 
the program, and the present task of the program returns to block 
118 to read the sensors again. 

20 Figure 5 is a representation of a video image, illustrating a 

method for controlling and assuring the integrity of the image 
upon processing, storage and playback. In applications where 
compression of the entire video image would be impossible and yet 
the integrity of the video image 80 must be preserved (such as in 

25 a real-time recording and playback system) , it is possible to 

sample and compress only a portion of the video signal from the 
CCD microcamera 22 and then to combine this signal with the 
signals from the various sensors 11, 12, 14, 16, 18, 20 and 21 
before encrypting the resulting data. The video image 80 can 

3 0 include an image area 82 and generally an area 84 in which the 
date, time and status of various sensors (SI, S2 and S3) are 
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displayed. In addition, both the entire video image 80 and. a 
subporcion 86 of the video image 80 are compressed. 

The entire video image 80 need not be encrypted before 
storage, since the subportion 8 6 of the video image 80 is 
5 encrypted and preserved for decryption and comparison to the 

stored image 70 to assure that the stored image is valid. Instead 
the entire video image 80 is converted to the file format but not 
combined with any sensor data. Rather, the subportion 86 is 
compressed and combined with the sensor data and then enci-ypted. 

10 Parameters giving the location of the subportion 8 6 are 

incorporated into the header data so that it will be possible to 
correctly compare the decrypted and decompressed subportion 80 
with the appropriate part of the decompressed entire video image 
80. The subportion 86 can be placed at any desired location in the 

15 entire video image 80. However, if desired, the location of the 
subportion can be changed randomly (say, to a. subportion £16') so 
that a potential counterfeiter would be unable to know which 
subportion 86, if any, of the entire image 80 was used to validate 
the played-back compressed entire video image 80. 

20 Figure 6 is a block diagram of an apparatus for playback of 

the data stored by the data recorder of the present invention. The 
playback apparatus 140 includes a playback unit 142, a decryption 
circuit 144, a decompression circuit 146, a playback signal 
processing circuit 148, a video display unit (VDU) 150, a speaker 

25 152 and a sensor display 154. The playback unit* 142 includes a 

control panel 155 from which a user of the playback apparatus 140 
can control the playback process, including entry of any data 
needed to facilitate the decryption process, such as entry of the 
second (unpublished) component of any public key encryption system 

30 that may be in use. The playback unit 142 accepts the removable 
hard disk that was removed from the mass storage device 3D, and 
plays back the removable hard disk to reproduce the digital data 
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stored on the removable hard disk. The signals representing the 
digital data are transmitted to the decryption circuit 144 which 
removes the effect of the encryption performed on the data prior 
to storage on the removable hard disk drive. The decryption 
5 circuit 144 produces a compressed signal and an uncompressed 

signal, including original sensor signals. The compressed signal 
is received by the decompression circuit 14 6 which produces a 
decompressed video signal and a decompressed sensor signal . The 
uncompressed signal from the decryption circuit 144 and both of 

10 the decompressed signals from the decompression circuit 146 are 
received by the signal processing circuit 148. In response, the 
signal processing circuit 148 produces an analog video signal and 
an audio signal. The analog video signal is received by the VDU 
150 which causes the VDU 150 to reproduce the original images. 

is Also, the speaker 152 receives the audio signal and reproduces the 
original audio signals that accompanied the original images. In 
addition, the sensor display 154 receives the sensor signals and 
reproduces their original readings. 

Figure 7 is a flow chart of the method of playing back the 

20 data stored by the data recorder of the present invention. In this 
method, the digital signals recorded on the removable hard disk 
are played back (block 160) . Next, the encrypted digital signals 
among the played-back signals are decrypted (block 162) . Any 
compressed signals included among those that were originally 

25 played back or later decrypted are then decompressed (block 164) 
and the resulting uncompressed and decompressed signals are 
processed to produce audio signals, video signals and sensor 
signals that were originally concurrent (block 166) . The audio 
signals are then played, and the concurrent video and sensor 

30 signals are displayed (block 168) , thereby reproducing the 

original video, audio and sensor signals. In the case where only a 
portion of the original video image is encrypted, the signal 
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processing circuit 148 also compares the decompressed decrypted 
subportion 86 of the entire video image 82 with the proper portion 
of the decompressed entire video image 82 before displaying the 
decompressed entire video image 82. If there are discrepancies, 
s the signal processing circuit 148 will alert the user to the type 
and location of the discrepancies. 

The preferred embodiment of the present invention has been 
described in detail sufficient for one skilled in the electronics 
io and radio frequency arts to understand the invention. Such skilled 
persons, however, could devise alternative embodiments to that 
described herein while remaining within the scope of the appended 
claims. Accordingly, the scope of the invention is to be limited 
only by the appended claims . 



• 
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Claims 



1 . A method for recording sensor data with a resultant 
5 video signal to protect the sensor data and the resultant video 
signal, the method comprising the steps of: 

a) selecting one or more sensors that produce the sensor 

data; 

b) transforming the signals from the one or more sensors 
10 to digital signals, including digitizing analog signals produced 

by some of the sensors and combining the digitized analog signals 
with digital signals produced by the remaining sensors to produce 
a digital sensor signal; 

c) selecting one or more video signal sources from a set 
15 of two or more video signal sources producing respective video 

signals; 

d) producing the resultant video signal from the 
respective video signals produced by the selected one or more 
video signal sources; 

20 e) encrypting at least one of the digital sensor signal 

and the resultant video signal in accordance with a digital 
encryption procedure to produce a respective digital output sensor 
signal and a respective digital output video signal; 

f) combining the respective digital output sensor signal 
25 and the respective digital output video signal to produce a 

digital output signal; and 

g) recording the digital output signal on a, recording 

medium . 



30 



2. The method of claim 1, wherein step e) includes 
encrypting only the digital sensor signal to produce a digital 
output sensor signal . 
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3. The method of claim 1, wherein step e) includes 
encrypting only the digital video signal to produce a digital 
output video signal. 

5 

4. The method of claim 1, wherein step c) includes 
selecting the video signal sources as a function of conditions. 

5. The method of claim 4, wherein the conditions include 

10 the values of the data produced by one or more of the sensors. 

j 

6. The method of claim 4, wherein the conditions include 
the values of the data produced by a sensor that measures a 
velocity . 

15 

7. The method of claim 4, wherein the conditions include 
the values of the data produced by a sensor that produces data 
representative of position. 

20 8. The method of claim 4, wherein the conditions include 

the values of the data produced by a sensor that is actuable by a 
human . 

9. The method of claim 4, wherein the conditions include 
25 the values of the data produced by a sensor that measures an 

acceleration . 

10. The method of claim 4, wherein the conditions include 
the values of the data produced by a sensor that measures an 

3 0 acceleration. 
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11. The method of claim 10, wherein the sensor that 
measures an acceleration measures accelerations in two different 
directions . 

5 12. The method of claim 10, wherein the sensor that 

measures an acceleration measures accelerations in two different 
directions . 

13. The method of claim 1, wherein step c) includes 

10 causing the one or more video signal sources to move so that the 
video signal sources produce signals representative of changeable 
directions . 

14. The method of claim 1, wherein step a) includes 

is selecting the sensor signal sources as a function of conditions. 

15. The method of claim 14 , wherein the conditions include 
the values of the data produced by one or more of the sensors . 

20 16. The method of claim 14, wherein the conditions include 

the values of the data produced by a sensor that measures a 
velocity. 

17. The method of claim 14, wherein the conditions include 
2 5 the values of the data produced by a sensor that produces data 

representative of position. 

18. The method of claim 14, wherein the conditions include 
the values of the data produced by a sensor that is actuable by a 

30 human. 
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19. The method of claim 14, wherein the conditions include 
the values of the data produced by a sensor that measures an 
acceleration . 

5 20. The method of claim 14, wherein the conditions include 

the values of the data produced by a sensor that measures an 
acceleration. 

21. The method of claim 20, wherein the sensor that 

10 measures an acceleration measures accelerations in two different 

J 

directions . 

22. The method of claim 20, wherein the sensor that 
measures an acceleration measures accelerations in two different 

is directions . 

23 . An apparatus for recording sensor data with a 
resultant video signal to protect the sensor data and the 
resultant video signal, the apparatus comprising: 

a selection circuit to select one or more sensors tnat 
produce the sensor data; 

a digitization circuit to transform the signals fro.n the one 
or more sensors to digital signals, the digitization circuit 
digitizing analog signals produced by some of the sensors and 
combining the digitized analog signals with digital signals 
produced by the remaining sensors to produce a digital ser.sor 
signal ; 

a selection circuit to select one or more video signal 
sources from a set of two or more video signal sources producing 
respective video signals; 
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a video production circuit to produce the resultant video 
signal from the respective video signals produced by the selected 
one or more video signal sources; 

an encryption circuit to encrypt at least one of the digital 
5 sensor signal and the resultant video signal in accordance with a 
digital encryption procedure and to produce a respective digital 
output sensor signal and a respective digital output video signal; 

a combination circuit to combine the respective digital 
output sensor 'signal and the respective digital output video 
10 signal to produce a digital output signal; and 

a recorder to record the digital output signal on a 
recording medium. 

24. The apparatus of claim 23, wherein the encryption 
is circuit encrypts only the digital sensor signal to produce a 

digital output sensor signal. 

25. The apparatus of claim 23, wherein the encryption 
circuit encrypts only the digital video signal to produce a 

20 digital output video signal. 

26. The apparatus of claim 23 r wherein the selection 
circuit selects the video signal sources as a function of 
conditions . 

25 

27. The apparatus of claim 26, wherein the selection 
circuit receives values of the data produced by one or more of the 
sensors . 

30 28. The apparatus of claim 26, wherein the conditions 

include the values of the data produced by a sensor that measures 
a velocity; 
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29. The apparatus of claim 26, wherein the conditions 
include the values of the data produced by a sensor that produces 
data representative of position. 

5 

30. The apparatus of claim 26, wherein the conditions 
include the values of the data produced by a sensor that is 
actuable by a human. 

10 31. The apparatus of claim 26, wherein the conditions 

include the values of the data produced by a sensor that measures 
an acceleration. 

32. The apparatus of claim 26, wherein the conditions 

is include the values of the data produced by a sensor that measures 
an acceleration. 

33. The apparatus of claim 32, wherein the sensor that 
measures an acceleration measures accelerations in two different 

20 directions. 

34. The apparatus of claim 32, wherein the sensor that 
measures an acceleration measures accelerations in two different 
directions . 



25 



35. The apparatus of claim 23, wherein step c) includes 
causing the one or more video signal sources to move so that the 
video signal sources produce signals representative of changeable 
directions . 



30 
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36. The apparatus of claim 23, wherein the selection 
circuit selects the sensor signal sources as a function of 
conditions . 

5 37. The apparatus of claim 36, wherein 

include the values of the data produced by one 
sensors . 



the conditions 
or more of the 



38. The apparatus of claim 36, wherein the conditions 
10 include the values of the data produced by a sensor that measures 
a velocity. 



39. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that produces 

is data representative of position. 

40. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that is 
actuable by a human. 

20 

41. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that measures 
an acceleration. 



42. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that measures 
an acceleration. 

43. The apparatus of claim 42, wherein the sensor that 
measures an acceleration measures accelerations in two different 
directions. 
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44. The apparatus of claim 42, wherein the sensor zhat 
measures an acceleration measures accelerations in two different 
directions . 



5 45. An apparatus for recording sensor data with a 

resultant video signal to protect the sensor data and the 
resultant video signal, the apparatus comprising; 

means for selecting one or more sensors that products the 
sensor data; 

10 means for transforming the signals from the one or more 

sensors to digital signals, the means for transforming the signals 
digitizing analog signals produced by some of the sensors and 
combining the digitized analog signals with digital signals 
produced by the remaining sensors to produce a digital sensor 

15 signal ; 

means for selecting one or more video signal sourceis from a 
set of two or more video signal sources producing respective video 
signals ; 

means for producing the resultant video signal from the 
20 respective video signals produced by the selected one or more 
video signal sources; 

means for encrypting at least one of the digital sensor 
signal and the resultant video signal in accordance with a digital 
encryption procedure and producing a respective digital output 
25 sensor signal and a respective digital output vixieo signal; 

means for combining the respective digital output sensor 
signal and the respective digital output video signal and 
producing a digital output signal therefrom; and 

means for recording the digital output signal on a recording 

30 medium. 
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46. The apparatus of claim 45, wherein the means for 
encrypting encrypts only the digital sensor signal to produce a 
digital output sensor signal . 

5 47. The apparatus of claim 45, wherein the means for 

encrypting encrypts only the digital video signal to produce a 
digital output video signal . 

48. The apparatus of claim 45, wherein the means for 
selecting selects the video signal sources as a function of 
conditions . 



10 



49. The apparatus of claim 48, wherein the means for 
selecting receives values of the data produced by one or more of 

is the sensors. 

50. The apparatus of claim 48, wherein the conditions 
include the values of the data produced by a sensor that measures 
a velocity. 

20 

51. The apparatus of claim 48, wherein the conditions 
include the values of the data produced by a sensor that produces 
data representative of position. 

25 52. The apparatus of claim 48, wherein the conditions 

include the values of the data produced by a sensor that is 
actuable by a human. 

53. The apparatus of claim 48, wherein the conditions 
30 include the values of the data produced by a sensor that measures 
an acceleration. 
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54. The apparatus of claim 48, wherein the conditions 
include the values of the data produced by a sensor that measures 
an acceleration. 

5 55. The apparatus of claim 45, wherein the means for 

selecting selects the sensor signal sources as a function of 
conditions . 



56. A method for recording sensor data with a video signal 
to prptect the sensor data, the sensor data being contained in 
signals from one or more sensors, the method comprising the steps 
of: 

a) encrypting the signals from the one or more sensors 

by: 

al) transforming the signals from the one or more 
sensors to digital signals, including digitizing analog signals 
produced by some of the sensors and combining the digitized analog 
signals with digital signals produced by the remaining sensors; 
and 

a2) encrypting the digital signals in accordance with 
a digital encryption procedure; 

b) combining the video signal and the encrypted signals 
to produce an output signal; and 

c) recording the output signal on a recording medium. 

♦ 

57. A method for retrieving sensor data originally 
produced by one or more sensors, a video signal and two or more 
signals representing the sensor data being encrypted to produce a 
recorded signal that protects the sensor data, the method 
comprising the steps of : 

a) playing back the recorded signal to produce a played- 
back signal ; 
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b) decrypting the played-back signal to produce a video 
signal and one or more sensor signals containing data originally 
produced by one or more sensors; and 

c) processing the one or more sensor signals to produce 
the sensor data. 

58. A method for retrieving sensor data originally 
produced by one or more sensors, a video signal and two or more 
signals representing the sensor data being encrypted to produce a 
recorded signal that protects the sensor data, the method 
comprising the steps of: 

a) playing back the recorded signal to produce a played- 
back signal; 

b) decrypting the played-back signal to produce a video 
signals and one or more sensor signals containing data originally 
produced by one or more sensors ; 

c) determining whether any of the one or more sensor 
signals are digital signals; and 

d) processing the one or more sensor signals to produce 
the sensor data. 

59. The method of claim 58, wherein at least one of the 
sensor signals is a digital signal that has been encrypted with a 
digital encryption procedure. 

60. The method of claim 59, wherein the digital encryption 
procedure is a public key encryption procedure. 

61. The method of claim 60, wherein step d) includes 
decompressing any digital sensor signals. 
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62. A method for recording sensor data with a video signal 
to protect the sensor data, the sensor data being contained in 
signals from one or more sensors, the method comprising the steps 

Of; 

a) determining which of the signals from the one or more 
sensors are analog sensor signals and which of the signals from 
the one or more sensors are digital sensor signals; 

b) converting the data in the analog sensor signals to 
one or more first digital signals; 

c) encrypting the digital sensor signals and the one or 
more first digital signals to produce second digital signaLs; 

d) converting the second digital signals to first analog 
signals; and 

e) recording the video signal and the first analog 
signals on a recording medium. 

63. The method of claim 62, wherein step c) comprises: 
cl) encrypting the digital sensor signals in accordance 

with a digital encryption procedure. 

64. The method of claim 63 # wherein step cl) includes 
encrypting the digital signals in accordance with a public key 
encryption procedure. 

25 65.. The method of claim 63, wherein step d) comprises: 

dl) compressing the second digital signals in accordance 
with a digital compression procedure. 

66 . A method for retrieving sensor data originally 
3 0 produced by one or more sensors, the sensor data being encrypted 

with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 
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sensors, at: least one of the sensors producing an analog signal, 
the method comprising the steps of: 

a) playing back the recorded signals to produce a played - 
back signal; 

b) decrypting the played-back signal to produce a video 
signals and one or more sensor signals containing data originally 
produced by one or more sensors ; 

c) determining whether any of the one or more sensor 
signals are digital signals; and 

d) processing the one or more sensor signals to produce 
the sensor data, including converting the one or more sensor 
signals to produce a corresponding analog signal . 

67. The method of claim 66, wherein at least one of the 
sensor signals is a digital signal that has been encrypted with a 
digital encryption procedure, and step d) including decrypting the 
digital signal according to the digital encryption procedure . 

68. The method of claim 67, wherein the digital encryption 
procedure is a public key encryption procedure. 

69. The method of claim 68, wherein step d) includes 
decompressing any digital sensor signals. 

70. Apparatus for recording sensor data with a video 
signal, the sensor data being contained in signals from one or 
more sensors, the apparatus comprising: 

an encryption circuit to encrypt the signals from the one or 
more sensors ; 

a signal processing circuit to combine the video signal and 
the encrypted signals to produce an output signal; and 
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a recorder to record the output signal on a recording 

medium. 

71. Apparatus for retrieving sensor data originally 

s produced by one or more sensors, the sensor data being encrypted 
with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 
sensors, the method comprising the steps of: 

a play-back circuit to play back the recorded signal to 
10 produce a played-back signal; 

a decryption circuit to decrypt the played-back signal to 
produce a video signal and one or more sensor signals containing 
data originally produced by one or more sensors; and 

a signal processing circuit to process the one or more 
is sensor signals to produce the sensor data. 

72. The apparatus of. claim 71, wherein at least one of the 
sensor signals is a digital signal that has been encrypted with a 
digital encryption procedure. 

20 

73. The apparatus of claim 71, wherein the digital 
encryption procedure is a public key encryption procedure. 

74 . Apparatus for recording sensor data with a video 
25 signal to protect the sensor data, the sensor data being contained 
in signals from one or more sensors, the apparatus comprising: 

a compression circuit to compress the video signal 
including : 

a digitizer to digitize the video signal to produce 
3 0 video digital signals therefrom; and 

a digital compression circuit to compress the video 
digital signals in accordance with a digital compression procedure 
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and produce the compressed video signal, the digital compression 
circuit including an encryption circuit to encrypt the digital 
signals in accordance with a digital encryption procedure; 

a signal processing circuit to combine the compressed video 
5 signal and the signals, from the one or more sensors and to produce 
a digital signal therefrom; 

an encryption circuit to encrypt the digital signal and 
produce an output signal therefrom; and 

a recorder to record the output signal on a recording 

10 medium. 

75. The apparatus of claim 74, wherein the encryption 
circuit includes a public key encryption circuit to encrypt the 
digital signals in accordance with a public key encryption 

is procedure. 

76. The apparatus of claim 74, wherein the video signal 
and the signals Containing the sensor data are digital signals, 
the compression circuit includes a digital compression circuit to 

20 compress the digital signals in accordance with a digital 

compression procedure, the signal processing circuit digitally 
combines the video signal and the encrypted signals, and the 
encryption circuit encrypts the digital signal in accordance with 
a digital encryption procedure. 

25 

77 . Apparatus for retrieving sensor data originally 
produced by one or more sensors, the sensor data being encrypted 
with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 

30 sensors, the apparatus comprising: 

a play-back circuit to play back the recorded signal to 
produce a played-back signal; 
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a decryption circuit to decrypt the played-back signal to 
produce a video signals and one or more sensor signals containing 
data originally produced by one or more sensors; 

a logic circuit to determine whether any of the one or more 
5 sensor signals are digital signals; and 

a signal processing circuit to process the one or rrore 
sensor signals to produce the sensor data. 

78. The apparatus of claim 77, wherein at least one of the 
10 sensor signals is a digital signal that has been encrypted with a 

digital encryption procedure. 

79. The apparatus of claim 78, wherein the digital 
encryption procedure is a public key encryption procedure. 

15 

80. The method of claim 78, wherein the signals processing 
circuit includes a decompression circuit to decompress any digital 
sensor signals. 



20 81 . Apparatus for recording sensor data with a video 

signal, the sensor data being contained in signals from one or 
more sensors, the apparatus comprising: 

a distinction circuit to determine which of the signals from 
the one or more sensors are analog sensor signals and which of the 
25 signals from the one or more sensors are digital sensor signals; 

a first conversion circuit to convert the data in the analog 
sensor signals to one or more first digital signals; 

an encryption circuit to encrypt the digital sensor signals 
and the one or more first digital signals to produce second 
30 digital signals; 

a second conversion circuit to convert the second digital 
signals to first analog signals; and 
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a recorder to record the video signal and the first analog 
signals on a recording medium. 

82. The apparatus of claim 81, wherein the encryption 
circuit comprises: 

a digital encryption circuit to encrypt the digital sensor 
signals in accordance with a digital encryption procedure. 

83. The apparatus of claim 82, wherein the digital 
encryption circuit includes a public key encryption circuit to 
encrypt the digital signals in accordance with a public key 
encryption procedure . 

84. The apparatus of claim 81, wherein the second 
conversion circuit includes a compression circuit to compress the 
second digital signals in accordance with a digital compression 
procedure . 

85. Apparatus for retrieving sensor data originally 
produced by one or more sensors, the sensor data being encrypted 
with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 
sensors, at least one of the sensors producing an analog signal, 
the apparatus comprising: 

a play-back circuit to play back the recorded signal to 
produce a played-back signal; 

a decryption circuit to decrypt the played-back signal to 
produce a video signal and one or more sensor signals containing 
data originally produced by one or more sensors : 

a logic circuit to determine whether any of the on sensor 
signals are digital signals; and 
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, a signal processing circuit to process the one or mo:re 
sensor signals to produce the sensor data, including converting 
the one or more sensor signals to produce a corresponding analog 
signal. 

5 

86. The apparatus of claim 85, wherein at least one of the 
sensor signals is a digital signal that has been encrypted with a 
digital encryption procedure, and the signal processing circuit 
includes a decryption circuit to decrypt the digital signal 

10 accordion to the digital encryption procedure. 

87. The apparatus of claim 86, wherein the digital 
encryption procedure is a public key encryption procedure. 

15 88. The apparatus of claim 87, wherein the decryption 

circuit decompresses any digital sensor signals. 

89. Apparatus for recording sensor data with a video 
signal to protect the sensor data, the sensor data being contained 

20 in signals from one or more sensors, the apparatus comprising: 
means for encrypting the signals from the one or mora 
sensors ; 

means for combining the video signal and the encrypt sd 
signals to produce an output signal; and 
25 means for recording the output signal on af recording medium. 

90. Apparatus for recording sensor data with a video 
signal to protect the sensor data, the sensor data being contained 
in signals from one or more sensors, the apparatus comprising: 

30 means for compressing the video signal to produce a 

compressed video signal; 
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means for combining the compressed video signal and the 
signals from the one or more sensors to produce a digital signal; 

means for encrypting the digital signal to produce an output 
signal; and 

5 means for recording the output signal on a recording medium. 

91. Apparatus for recording sensor data with a video 
signal to protect the sensor data, the sensor data being contained 
in signals from one or more sensors, the apparatus comprising: 
io means for determining which of the signals from the one or 

more sensors are analog sensor signals and which of the signals 
from the one or more sensors are digital sensor signals ; 

first means for converting the data in the analog sensor 
signals to one or more first digital signals; 
is . means for encrypting the digital sensor signals and the one 

or more first digital signals to produce second digital signals; 

second means for converting the second digital signals to 
first analog signals; and 

means for recording the video signal and the first analog 
20 signals on a recording medium. 

92 . A method for recording sensor data with a video 
signal, the sensor data being contained in signals from one or 
more sensors, the method comprising the steps of: 
25 a) encrypting the signals from the one or more sensors to 

produce an output signal ; and 

b) recording the output signal and the video signal on a 
recording medium. 

30 93 . Apparatus for recording sensor data with a video 

signal, the sensor data being contained in signals from one or 
more sensors, comprising: 
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an encryption circuit to encrypt the signals from the one or 
more sensors to produce an output signal ; and 

a recorder to record the output signal and the video signal 
on a recording medium. 

5 

94. A method for recording sensor data with a video signal 
in a motor vehicle, the sensor data being contained in signals 
from one or more sensors in the motor vehicle, the method 
comprising the steps of: 
io a) encrypting the signals from the one or more seniors; 

b) combining the video signal and the encrypted si'jnals 
to produce an output signal; and 

c) recording the output signal on a recording medium. 

is 95. Apparatus for recording sensor data with a video 

signal in a motor vehicle, the sensor data being contained in 
signals from one or more sensors in the motor vehicle, the 
apparatus comprising : 

an encryption circuit to encrypt the signals from tho one or 
20 more sensors; 

a circuit to combine the video signal and the encrypted 
signals and produce an output signal therefrom; and 

a recorder to record the output signal on a recording 

medium. 

25 • 
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(57) Abstract 



A method and apparatus for recording sensor data (1 1-21). The analog and digital signals containing the sensor data (1 1-21) are 
accompanied by a concurrent analog video signal (22. 25). The analog signals are converted to digital form and highly redundant signals 
are compressed according to conventional compression techniques (36). The resulting compressed and uncompressed signals are encrypted 
and stored on a removable hard disk (30). The data stored on the removable hard disk (30) can later be played back to reconstruct the 
original signals while assuring that the played back signals are correct reconstructions of the original signals. If desired, the data collected 
can be made a function of one or more of the sensors (11-21). Also, the data can be transmitted to a remote location or control signals can 
be received from a remote location. 
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Description 

METHOD AND APPARATUS FOR RECORDING AND REPRODUCING SENSOR DATA 

Technica l Field 

The present invention relates to methods and apparatus for 
recording and reproducing signals and data and f more particularly, 
to methods and apparatus for recording and reproducing sensor data 
and video signals. 



Background of the Invention 

Motor vehicles (including personal automobiles, transit buses 
and transit trains) are an integral part of modern societies. 
Beside providing the facility of transporting goods in the market 
15 place, they also transport millions of people on a daily basis and 
are used to provide general services, such as public safety 
services. The burgeoning use of motor vehicles has given rise to a 
complex and expensive liability system concerning vehicular 
transportation. In nearly all states of the United States, for 
example, automobile insurance is required. Accordingly, much 
expensive litigation ensues in connection with motor vehicle 
accidents and other incidents. 

The massive system of policies and claims that result from 
the insurance coverage and related litigation are subject to 
25 fraudulent abuse. Current estimates indicate that this phenomenon 
is on the rise, causing increased cost to the system, even in 
locales where accident rates are diminishing. In addition to 
fraud, the resolution of claims relating to motor vehicles is an 
inefficient time-consuming process that experiences additional 
costs by relying heavily on a tort-based legal process. As one 
measure of the increased burden that this phenomenon puts on 
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society, legal costs concerning automotive and transit bus injury 
claims is increasing significantly. 

One problem related to these increased costs is the 
difficulty in providing basic documented facts concerning motor 
5 vehicle (e.g., automobile and transit bus) incidents. Beside the 
abuse of the process allowed by parties bringing suit in the hopes 
of an out-of-court settlement, costs are often significantly 
greater when a motor vehicle-related suit does end up in court. To 
combat these problems of documentation and out -of -control costs, 

10 many insurance companies and states in the United States are 

considering implementation of mandatory vehicle inspections and 
special investigation personnel units. However/ these approaches 
are expensive, too, and this expense adds to the overall costs of 
the transportation system. 

15 One approach to helping curtail these costs is to provide 

better data concerning motor vehicle incidents. This will help to 
curtail fraudulent claims . While other vehicle-borne systems have 
been proposed to provide information concerning automotive and 
transit bus accidents, these systems are themselves subject to 

20 fraudulent manipulation. As one result, courts of law are not 

likely to give much evidential weight to such information systems 
unless they are secure. 

Accordingly, it is desirable to have a vehicle-borne data 
collection system which is capable of securely recording data 

25 concerning accidents. Of course, such systems wi^.1 have other 
applications, such as other means of transportation and other 
situations demanding secure preservation of documentary data. 
Other applications concerning automotive transportation include 
taxis, limousines and rental vehicles . Depending upon the 

3 0 circumstances, the documentary data can include video data (and 

accompanying audio data) concerning occurrences exterior to and/or 
within the automotive transportation unit, as well as data related 
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to operation of the unit such as dynamic variables (for example, 
position, velocity and acceleration) and data provided by other 
analog and digital sensors . 

Examples of alternative transportation systems (aside from 
automotive transportation units) include transit buses and 
trolleys, subway and rail cars, air- and water-borne craft and 
elevators. Other, non- transportation systems include stationary 
systems, such as bank, other building, and location (such as ATM 
location) security systems. 

In another area of applications, the increasingly litigious 
societal environment has led to an increased need for a 
transportable way to document the circumstances of other (i.e., 
other than motor vehicle) incidents. Examples of such incidents 
include incidents that should be documented by police and other 
15 public safety officers as well as non-governmental persons such as 
insurance and security agents and other technicians. Such 
documentation is typically accomplished through the use of 
recorded still and moving images as well as outputs of other 
analog and digital sensors. As electronics is continues to have 
20 even further impact on photographic and videographic processes, it 
is natural that photographic and videographic equipment be used 
for these documentary purposes. As described above, there is a 
definite need to document and validate the images and sounds taken 
when doing the documentation. r 
25 Regardless of whether the video and/or sensor data are 

collected by a system that has components in potentially insecure 
locations, an additional level of data security can be provided by 
transmitting (or transporting) the data to a remote location. 
Aside from the additional security provided, transmitting the data 
to a remote location allows the data to be processed as it is 
being collected. In some circumstances, this processing can be 
used to assure system managers that the video sources and/or 
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sensors are operating properly. In other circumstances, remote 
processing of data can be used to determine whether the data 
should be collected in some other manner or form, such as by 
repositioning the video sources (possibly by means of control 
5 signals transmitted through a reverse transmission link) . 

Summary of the Invention 

According to one aspect, the invention is a method for 
recording sensor data with a resultant video signal to protect the 

10 sensor data and the resultant video signal. The method includes 
the steps of a) selecting one or more sensors that produce the 
sensor data and b) transforming the signals from the one or more 
sensors to digital signals. The method further includes the steps 
of c) selecting one or more video signal sources from a set of two 

is or more video signal sources producing respective video signals 
and d) producing the resultant video signal from the respective 
video signals produced by the selected one or more video signal 
sources . 

In addition, the method includes the steps of e) encrypting at 
20 least one of the digital sensor signal and the resultant video 
signal in accordance with a digital encryption procedure to 
produce a respective digital output sensor signal and a respective 
digital output video signal, f) combining the respective digital 
output sensor signal and the respective digital output video 
25 signal to produce a digital output signal and g) ( recording the 
digital output signal on a recording medium. Step a) includes 
digitizing analog signals produced by some of the sensors and 
combining the digitized analog signals with digital signals 
produced by the remaining sensors to produce a digital sensor 
30 signal. 

According to a further aspect, the invention is an apparatus 
for recording sensor data with a resultant video signal to protect 
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the sensor data and the resultant video signal. The apparatus 
comprises a selection circuit to select one or more sensors that 
produce the sensor data and a digitization circuit to transform 
the signals from the one or more sensors to digital signals. The 
digitization circuit digitizes analog signals produced by some of 
the sensors and combines the digitized analog signals with digital 
signals produced by the remaining sensors to produce a digital 
sensor signal. The invention further includes a selection circuit 
to select one or more video signal sources from a set of two or 
more video signal sources producing respective video signals and a 
video production circuit to produce the resultant video signal 
from the respective video signals produced by the selected one or 
more video signal sources. In addition the invention includes an 
encryption circuit to encrypt at least one of the digital sensor 
signal and the resultant video signal in accordance with a digital 
encryption procedure and to produce a respective digital output 
sensor signal and a respective digital output video signal, a 
combination circuit to combine the respective digital output 
sensor signal and the respective digital output video signal to 
produce a digital output signal and a recorder to record the 
digital output signal on a recording medium. 

According to one further aspect, the invention is a method 
for recording sensor data with a video signal to protect the 
sensor data. The sensor data is contained in signals from bne or 
more sensors. The method comprises the steps of a) encrypting the 
signals from the one or more sensors, b) combining the video 
signal and the encrypted signals to produce an output signal, and 
c) recording the output signal on a recording medium. 

According to another further aspect, the invention is a 
method for retrieving sensor data originally produced by one or 
more sensors. The sensor data is encrypted with a video signal to 
produce a recorded signal that protects the sensor data and being 
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contained in signals from one or more sensors. The method 
comprises the steps of a) playing back the recorded signal to 
produce a played-back signal, b) decrypting the played-back signal 
to produce a video signal and one or more sensor signals 
5 containing data originally produced by one or more sensors, and c) 
processing the one or more sensor signals to produce the sensor 
data . 

According to still another further aspect, the invention is a 
method for recording sensor data with a video signal to protect 

10 the sensor data. The sensor data is contained in signals from one 
or more sensors. The method comprises the steps of a) compressing 
the video signal to produce a compressed video signal, b) 
combining the compressed video signal and the signals from the one 
or more sensors to produce a digital signal, c) encrypting the 

15 digital signal to produce an output signal, and d) recording the 
output signal on a recording medium. 

According to yet another further aspect, the invention is a 
method for retrieving sensor data originally produced by one or 
more sensors. The sensor data is encrypted with a video signal to 

20 produce a recorded signal that protects the sensor data and being 
contained in signals from one or more sensors, at least one of the 
sensors producing an analog signal. The method comprises the steps 
of a) playing back the recorded signals to produce a played-back 
signal, b) decrypting the played-back signal to produce a video 

25 signals and one or more sensor signals containing data originally 
produced by one or more sensors, c) determining whether any of the 
one or more sensor signals are digital signals, and d) processing 
the one or more sensor signals to produce the sensor data, 
including converting the one or more sensor signals to produce a 

30 corresponding analog signal. 

According to a still further aspect, the invention is a 
method for recording sensor data with a video signal to protect 
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the sensor data. The sensor data is contained in signals from one 
or more sensors. The method comprises the steps of a) determining 
which of the signals from the one or more sensors are analog 
sensor signals and which of the signals from the one or more 
sensors are digital sensor signals, b) converting the data in the 
analog sensor signals to one or more first digital signals, c) 
encrypting the digital sensor signals and the one or more first 
digital signals to produce second digital signals, d) converting 
the second digital signals to first analog signals, and e) 
recording the video signal and the first analog signals on a 
recording medium. 

According to yet another aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 
is or more sensors. The apparatus comprises an encryption circuit, a 
signal processing circuit and a recorder. The encryption circuit 
encrypts the signals from the one or more sensors. The signal 
processing circuit combines the video signal and the encrypted 
signals to produce an output signal, and the recorder records the 
20 output signal on a recording medium. 

According to yet another aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 
or more sensors. The apparatus comprises a compression circuit, a 
25 signal processing circuit, an encryption circuit and a recorder. 
The compression circuit compresses the video signal. The signal 
processing circuit combines the compressed video signal and the 
signals from the one or more sensors and produces a digital signal 
therefrom. The encryption circuit encrypts the digital signal and 
produces an output signal therefrom. The recorder records the 
output signal on a recording medium. 
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According to an even further aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 
or more sensors. The apparatus comprises a distinction circuit, a 
5 first conversion circuit, an encryption circuit, a second 
conversion circuit, and a recorder. The distinction circuit 
determines which of the signals from the one or more sensors are 
analog sensor signals and which of the signals' from the one or 
more sensors are digital sensor signals. The first conversion 

10 circuit converts the data in the analog sensor signals to one or 
more first digital signals. The encryption circuit encrypts the 
digital sensor signals and the one or more first digital signals 
to produce second digital signals. The second conversion circuit 
convert the second digital signals to first analog signals-, and 

is the recorder records the video signal and the first analog signals 
on a recording medium. 

According to even still another aspect, the invention is an 
apparatus for recording sensor data with a video signal to protect 
the sensor data. The sensor data is contained in signals from one 

20 or more sensors. The apparatus comprises means for encrypting, 
means for combining and means for recording. The means for 
encrypting encrypts the signals from the one or more sensors. The 
means for combining combines the video signal and the encrypted 
signals to produce an output signal. The means for recording 

25 records the output signal on a recording medium. 

According to another aspect, the invention is a method for 
retrieving sensor data originally produced by one or more sensors, 
the sensor data being encrypted with a video signal to produce a 
recorded signal that protects the sensor data and being contained 

30 in signals from one or more sensors, at least one of the sensors 
producing an analog signal. The method comprises the steps of a) 
playing back the recorded signals to produce a played-back signal, 
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b) decrypting the played-back signal to produce a video signals 
and one or more sensor signals containing data originally produced 
by one or more sensors, c) determining whether any of the one or 
more sensor signals are digital signals, and d) processing the one 
or more sensor signals to piroduce the sensor data, including 
converting the one or more sensor signals to produce a 
corresponding analog signal . 

According to and even still further aspect, the invention is 
an apparatus for recording sensor data with a video signal to 
protect the sensor data. The sensor data is contained in signals 
from one or more sensors. The apparatus comprises means for 
compress, means for combining, means for encrypting and means for 
recording. The means for compressing compresses the video signal 
to produce a compressed video signal. The means for combining 
15 combines the compressed video signal and the signals from the one 
or more sensors to produce a digital signal . The means for 
encrypting encrypts theMigital signal to produce an output 
signal, and the means for recording records the output signal on a 
recording medium. 

In still yet another aspect, the invention is an apparatus 
for recording sensor data with a video signal to protect the 
sensor data. The sensor data is contained in signals from one or 
more sensors. The apparatus comprises means for determining, first 
and second means for converting, means for encrypting, and means 
25 for recording. The means for determining determines which of the 

signals from the one or more sensors are analog sensor signals and 
which of the signals from the one or more sensors are digital 
sensor signals. The first means for converting converts the data 
in the analog sensor signals to one or more first digital signals. 
30 The means for encrypting encrypts the digital sensor signals and 
the one or more first digital signals to produce second digital 
signals. The second means for converting converts the second 
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digital signals to first analog signals, and the means for 
recording records the video signal and the first analog signals on 
a recording medium. 

5 Brief Description of the Drawings 

Figure 1 is a schematic diagram of a first preferred 
embodiment of an apparatus of the invention. 

Figure 2 is a schematic diagram of the structure of a data 
file, as stored by the preferred embodiments of the invention. 
10 Figure 3 is a schematic diagram of a second preferred 

embodiment of an apparatus of the invention. 

Figure 4 is a flow chart of the program running within the 
preferred embodiments of the invention. 

Figure 5 is a representation of a video image, illustrating a 
is method for controlling and assuring the integrity of the image 
upon processing, storage and playback. 

Figure 6 is a block diagram of an apparatus for playback of 
the data stored by the data recorder of the present invention. 

Figure 7 is a flow chart of the method of playing back the 
20 data stored by the data recorder of the present invention. 

Detailed Description of the Invention 

Figure 1 is a schematic diagram of a preferred embodiment of 
an apparatus of the invention. The sensor data recorder 10 can be 

25 adapted for placement at or near the location wh^re the sensors 
are located. In the following detailed description, it will be 
assumed that the data recorder 10 is located in an automobile (not 
shown) . Within the automobile, the data recorder 10 can be placed 
in a passenger compartment or, for greater security, can be placed 

30 in a more secure place such as a trunk. Of course, those skilled 
in the art would provide appropriate protection for the data 
recorder 10, depending upon the particular task to which the data 
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recorder 10 is applied. For example, a data recorder 10 that is to 
be installed in an automobile should probably be shielded from 
accelerations due to impacts, as well as temperature extremes and 
high humidity. Other applications will obviously have their own 
particular environmental factors to be considered in the 
installation. 

In the preferred embodiment, the data recorder 10 includes a 
number of sensors, such as ignition switch 11, brake sensor 12, 
accelerator sensor 14, steering column sensor 16, wheel rotation 
sensor 18, turn signal sensor 20 and ultrasound detection sensor 
21. The data recorder 10 also includes a source of video 
information such as a forward- facing charge -coupled device (CCD) 
microcamera 22 and a recorder 24. The data recorder 10 may also 
include one or more optionally additional CCD microcameras, such 
as CCD microcamera 25 that can be pointed rearward. Of course, in 
non-automotive applications the sensors 11, 12, 14, 16, 18, 20 and 
21 can be augmented or substituted by other sensors that are 
dependent upon the application of the data recorder 10. For 
example, the sensors can include one or more microphones 13 and a 
sensor 15 (such as a global position satellite, GPS, sensor) 
capable of measuring dynamic variables such as position and 
velocity to sufficient accuracy for the task of the data recorder 
10. The sensors can also include one or more simple switches 17 
(possibly contained in a keyboard) for inputs by a human ' 
25 observer/user . 

The CCD microcamera 22 can be a conventional camera mounted 
on a universal mount. Appropriate specifications are that the CCD 
microcamera 22 should be operable in an operating temperature 
range from -10 Celsius to +60 Celsius and to a maximum relative 
humidity of 95%. Further, the CCD microcamera 22 should have 
automatic shutter sensitivity ranging from 1/60 to 1/32000 second 
and a resolution of at least 380 by 380 picture element (pixel) 
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rows and columns. In addition, the CCD microcamera 22 should have 
an illumination sensitivity of 1 lux or less. 

In some embodiments, it may be desirable to place the CCD 
microcamera 22 on a pan/tilt mount in a weatherproof enclosure. 
5 The CCD microcamera 22 could then be mounted outside of a vehicle 
and equipped with radio, infrared or ultrasound beacon- following 
sensors. Implementation of such sensors are well within the skills 
of those of the relevant electronic arts. Such beacon- following 
systems can be used to follow and provide video images of a person 

10 carrying an appropriate beacon source. An important example is a 
microcamera adapted to follow public safety personnel who are 
wearing a beacon and whose motions and surroundings can then be 
automatically recorded in video form. Further, if a radio or 
infrared beacon is used, such a system could also provide an audio 

is signal from the vicinity of the person wearing the beacon. The 
audio signal could carry the voice of the person wearing the 
beacon, and could be compressed, encrypted and recorded with the 
data from the other sensors, as will be described below. 
Alternatively, the pan/tilt mount can be controlled by a motor 

20 drive that receives control signals from external sources, as will 
be described in further detail subsequently. 

The recorder 24 can be a conventional JVC time lapse "cassette 
recorder, although other recorders (as described subsequently) 
could also be used. 

25 The sensors 11, 12, 13, 14, 15, 16, 17, 18, ( 20 and 21, and 

the CCD microcamera 22 (and CCD microcamera 25, if included) are 
connected to the recorder 24. Also, a display 23 can be connected 
to the recorder 24 (for example, to the sensor I/O circuitry 50) 
for use by a human user /operator . 

30 The various sensors 11, 12, 13, 14, 15, 16, 17, 18, 20 and 21 

may include a local transducer at the location of the variable 
being sensed and a remote transducer placed more closely to the 
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recorder 24 than the location of the variable being sensed. For 
example, the wheel rotation sensor 18 may include a local 
transducer that transforms some of the rotational energy into 
electrical energy and a remote transducer that converts the 
electrical energy into an electromagnetic signal that can be 
recorded by the recorder 24. The local transducer of the wheel 
rotation sensor 18 may be a segmented ferrous -based cog wheel that 
rotates with the wheel and the remote transducer can be a 
reluctance- sensing transducer that transforms the detected pulses 
in the magnetic field to an electrical signal that is processed 
for optimal use by the data recorder 10. 

Similarly, the ultrasound detection sensor 21 can have 
forward- or side-located components that produce signals that are 
further processed at a remote location. In one embodiment, the 
15 ultrasound detection sensor 21 can include a plurality of 

ultrasonic transducers mounted at the front and around the sides 
of the vehicle. From those locations, transducers produce and/or 
receive ultrasonic (or electromagnetic) signals to sense or 
determine the immediate surroundings of the vehicle and produce 
signals that can be processed to produce an alarm signal that will 
activate the inventive data recorder 10 to record the proximity of 
other nearby vehicles. Depending upon the particular signal 
processing used, in one embodiment such an ultrasound detection 
sensor 21 can be used to monitor the driving habits or the 
25 vehicle's operator ( s) . 

The recorder 24 shown in Figure 1 is a hardware -based 
implementation of an embodiment of the invention. In general, the 
recorder 24 includes a mass storage device 30, a bus 32, a signal 
processing circuit 34, a signal conversion circuit 3 6 and a sensor 
input/output (I/O) circuit 50. The mass storage device 30, the 
signal processing circuit 34, and the signal conversion circuit 
36, and the sensors I/O circuit 50 are all connected to the bus 
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32. The mass storage device 30 can take several forms. In one 
preferred embodiment the mass storage device 30 is a magnetic tape 
recorder (specified above) that records on a magnetic tape 
included in a removable cassette. In other embodiments, the mass 
5 storage device 3 0 can be a solid-state flash memory card or other 
removable memory mediums that store data. Examples are 2.5 and 3.5 
inch removable disk drives capable of storing at least 120 
megabytes of data. 

The signal processing circuit 34 includes a central 

10 processing unit (CPU) 40, a random access memory (RAM) 42 and a 
clock 44 that are connected together in conventional fashion. 
These connected components allow the CPU 40 to process sensor and 
other signals received from the bus 32 in accordance with clock 
signals produced by the clock 44 and to record the results of the 

15 signal processing operations in the RAM 42. The actual signal 
processing performed by the CPU 40 is dependent upon a program 
which is installed in the CPU 40 when the data recorder 10 is 
first turned on. Such programming is conventional to those skilled 
in the art of computer programming. Two aspects of the necessary 

20 programming are worthy of specific note, however. The first is 

that the CPU 40 is programmed to convert digital signals from one 
format to another. The second is that the CPU 40 is also 
programmed to encrypt the information in the signals. The actual 
encryption method can be any one of many available possibilities. 

25 The preferred form, however, is a public key encryption method 
chosen from the many known public key encryption methods. 

The signal conversion circuit 36 is used to convert the video 
signal from the CCD microcamera (s) 22 (and 25) . At a minimum, the 
signal conversion circuit 3 6 must convert the analog video signal 

30 to digital data. Further, the signal conversion circuit 3 6 can 

control the sequencing of the video signals if there is more than 
one microcamera. The signal conversion circuit 36 can include a 
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multiplexer that controls which CCD microcamera is being recorded 
at a given time. As will be described below, the rate at which the 
video signals are switched can be made a function of the state of 
alarm of the recorder 10. Accordingly, the signal conversion 
circuit 36 can be made to respond to alarm signals produced by the 
signal processing circuit 34. Alternatively, the signal conversion 
circuit 3 6 can take video signals from all of the CCD microcameras 
in operation and produce a concurrently composite video signal, 
such as a spli£ screen video signal. The signal conversion circuit 
36 can also receive date and time signals from the signal 
processing circuit 34 and produce video representations of those 
date and time signals in the analog video signal before digitizing 
the video signal . 

In some versions of the preferred embodiment of Figure 1, the 
signal processing circuit 34 can advantageously be responsive to 
the outputs of one or more of the sensors 11, 12, 13, 14, 15, is, 
17, 18, 19, or 20. As an important example, if the recorder 10 is 
installed in a transit bus, it may be advantageous to have four 
microcameras (such as microcameras 22 and 26) connected, all 
directed toward the interior of the transit bus. Two of the 
microcameras can be located at the front and rear ends of the bus 
and the other two microcameras can be located to monitor the 
transit bus' entry/exit doors. The signal processing circuit 34 
can be caused to switch from the fore- and aftward pointing 
microcameras to the door-monitoring microcameras as a function of 
the position of switches 17 which can be connected to the 
entry/exit doors in such a way that determines whether the doors 
are closed. When the switches 17 indicate that the entry/exit 
doors are closed, the signal processing circuit 34 causes the 
video signals from the fore-and aftward pointing microcameras to 
be used, while the door monitoring microcameras are used when the 
doors are open . 



-15- 



BNSDOCID: <WO 9723075A 1_IA> 



WO 97/23075 



PCT/US96/20912 



As a further embellishment of the transit bus system just 
described, the microcameras whose signals are being recorded at 
any time can be a function of time or location (via the GPS sensor 
15) . Without limitation, such a system can be used to monitor 
5 events exterior to the transit bus when the bus is passing through 
a high crime area. Also such a system can be used to monitor 
specific areas with the microcameras in response to an emergency 
event signified by actuation of a switch 17 (by the transit bus 
operator, for example) or by actuation of the accelerator sensor 
10 14 . It is possible to use the accelerator sensor 14 to sense the 
occurrence of a collision when the acceleration levels sensed 
exceed a predetermined threshold. 

Further, when desired, it is possible designate special 
events which will be preserved as inviolate data to be recorded by 
is the recorder 24. As an example, it is possible that the transit 
bus operator may want to record events that occur within and/or 
exterior to the transit bus and to preserve those events without 
fear that they will subsequently be written over inadvertently. 
Typically, such a recording will include a fixed predetermined 
o period of time, including periods of time before and after the 
switch 17 is actuated. In one system, the first actuation of the 
switch 17 can specify that a recordable event has occurred and the 
next subsequent actuation of the switch 17 can specify the end of 
the event. The recorder 24 can then adjust the portion of the 
5 video and sensor signals that it preserves. 

In addition, the signal conversion circuit 36 compresses the 
highly redundant video signals from the CCD microcamera 22 and 
produces therefrom a compressed digital signal that it transfers 
to the bus 32. The compression operation is performed by hardware 
o in the signal conversion circuit 36. This hardware conventionally 
takes the form of an application-specific integrated circuit 
(ASIC) that performs JPEG compression. Alternatively, and in some 
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applications, it may be preferable to perform compression of the 
video data through other hardware means, such as an ASIC that 
performs MPEG or other well-known compression techniques. 
Specifically, the compression portion of the conversion circuit 36 
5 compresses data received from the CCD microcamera 22, as well as 
any of the optional additional CCD microcameras 25 that may be in 
use in the specific preferred embodiment that is suitable for a 
particular application. It is also possible to perform a 
simultaneous cpmpression and encryption operation at this stage of 
10 the signal conversion process. 

In operation, the recorder 24 receives signals from the 
sensors 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 and 21, through the 
sensor I/O circuit 50 and the bus 32. The recorder 24 also 
receives signals from the CCD microcamera 22 (and any possible 
is optional additional microcameras 25) through the signal conversion 
circuit 36. The signal processing circuit 34 receives the sensor 
and digital video signals through the bus 32, processes the 
signals, and sends the resulting processed digital signals to the 
mass storage 3 0 for recording. 
20 Since the data recorded in the mass storage is encrypted, its 

security during playback can be given a very high degree of 
assurance. If a party wished to alter the data recorded in the 
mass storage, the encryption system applied would have to be 
defeated and the recorded data recorded over with counterfeit data 
25 that make contextual sense. In general, it would be very difficult 
to do so. It would be impossible, even given the massive amount of 
computing power available today, to counterfeit the data that can 
be stored by the present invention if a time constraint is 
applied, even a time constraint measured in years. 
30 In some applications, it may be desirable to omit the signal 

conversion circuit 36 and transfer the data from the CCD 
microcamera 22 directly to the bus 32. In other applications, it 
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may be desirable to omit the encryption of the digital data 
received by the signal processing circuit 34 . 

As applied to one particular desired specific embodiment of 
the invention, the signals from the CCD microcamera 22 can be 
5 compressed by the compression circuit (such as a JPEG circuit) . 
Then the compressed data, along with the sensor data can be mixed 
with clock data before encryption, so that the encrypted data 
includes contemporaneous recordings of the date and time (clock 
signal), the sensor positions (sensor data) and the visual scene 

10 (video data) . If desired, the date and time information can be 

inserted into the visual scene (through conventional circuitry and 
processing, not shown) . 

It may be desirable to retain remote copies of the data 
collected by the recorder 24 . In some versions of the preferred 

is embodiment, it may be the primary function of the recorder 24 to 
transmit the data to a remote location, while retaining a copy of 
the data transmitted for backup or archive purposes. In addition 
to storing and/or processing the data, such a system can provide 
control signals to the recorder 10 to alter the response and/or 

20 configuration of the recorder 10. To accomplish this, the recorder 
10 can also include a controller 200, a transceiver 202, and an 
antenna 2 06 that communicates with a remote transceiving 
system/ computer 208 . The controller 200 is connected to the bus 
32, the transceiver 204 is connected to the controller 200, and 

25 the antenna 206 is connected to the transceiver ,204 . When data 
collected, compressed and encrypted by the recorder 10 is to be 
transmitted to the remote location, the data are prepared by the 
recorder 10 and then sent to the controller 200 over the bus 32. 
the signals representing the data to be transmitted are augmented 

3 0 by appropriate control signals, to which the controller 200 

responds. The controller 200 then controls the transceiver 204 and 
causes it to transmit the data to the remote location through the 
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antenna 206. Of course, it would be well-known to those skilled in 
the art to transmit to the remote location via electromagnetic 
energy . It would also be well-known to use ultrasonic radiation 
to transmit signals to a remote location. 

After the data are received at the remote location, the 
remote transceiving system/computer 208 can be configured to* 
record and process the data. Depending upon the results of the 
data processing, it may be desirable to transmit control signals 
from the remote transceiving system/computer 208 to the recorder 
10. To accomplish this, the control signals are transmitted to the 
antenna 206 by the remote transceiving system/computer 208, the 
received signals being demodulated by the transceiver 204 and 
passed on to the controller 200 for further processing. Any data 
included with the data signals can be separated by the controller 
is 200, which then responds to the control signals and causes the 

data to be transmitted to the appropriate portion of the recorder 
10. As a simple example, the remote transceiving system/computer 
208 may produce control signals to activate a sensor or to cause 
one or the other of the microcameras 22 and 25 to be repointed to 
20 collect video data from a desired direction. 

Figure 2 is a schematic diagram of the structure of a data 
file, as stored by the preferred embodiment of the invention. The 
data file 60 is represented as a rectangular area containing 
digital data which are stored in a memory such as the RAM ' 42 of 
25 the signal processing circuit 34 in Figure 1. However, the data 
have a format which is useful when reading and playing back the 
data. Specifically, the data file contains header data 62, video 
data 64 and sensor data 66, in that order. The header data 62 
includes an indication of the file format that is used in the 
storage of the data in the file, the version of the recorder on 
which the data were recorded, a customer identification number, 
and additional portions reserved for future use. The video data 64 



30 



-19- 



BNSDOCID: <WO 9723075A1_1A> 



WO 97/23075 



PCT/US96/20912 



is 8-bit gray scale data, representing 256 different levels of 
gray. Alternatively, the video data 64 can represent a color video 
image. The video data 64 can be postprocessed, after the analog 
video signals have been digitized, by the insertion of a special 
5 digital data portion 68 that contains documentary data. Most often 
the documentary data will be the date and time at which the images 
in the video data were imaged by the microcamera (s) 22 and 25. 
However, the documentary data could be expanded to include 
digitized representations of the outputs of some or all of the 

10 sensors 11, 12, 14, 16, 18, 20 and 21. 

While this procedure may document the video data 64, it can 
be useful to provide a first level of security to the video data 
by reserving specific pixels represented by the video data 64 
which are used by the fonts of the characters used to represent 

is the documentary data in the digital data portion 68. The font is 
chosen so that it is very difficult to reproduce if an attempt is 
made to modify the video data 64, but is very easy to verify as 
valid if no attempt to modify the video data 64 has been made. 

The sensor data 66 includes digital representations of the 

20 value of the variable (s) for each of the sensors 11, 12, 14, 16, 
18, 20 and 21. Typically, it is desirable to encode the digital 
sensor data 66 to discourage tampering. However, the primary 
security for the sensor and other data in the file format is 
provided by encryption that is applied after the data in the file 

25 are compressed. 

There are further procedures to heighten the security of the 
data, particularly the video data 64 stored in the file format 60. 
One approach is to encode the gray scale (or color) information in 
the file format 60 so that consecutive digital values do not 

30 represent consecutive grays or colors. In this way, the data are 
represented in a false gray (or color) fashion that will 
discourage tampering . 
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Figure 3 is a schematic diagram of a second preferred 
embodiment of an apparatus of the invention. The recorder 24 shown 
in Figure 3 is a software-based implementation of an embodiment of 
the invention. In this schematic diagram, components which serve 
5 the same purpose as those described in Figure l are given the same 
reference numerals as their counterparts in Figure 1 . 

In Figure 1, ignition switch sensor 11 may take the form of a 
conventional ignition switch that can be accessed so that it is 
possible to determine whether the ignition switch is turned on. 
10 Turning on the ignition is generally a good signal that the 

vehicle is about to be operated and may, in some system, signify 
that the data recording process should begin. . In addition, the 
brake sensor 12 may take the form of a brake pressure sensor, the 
accelerator sensor 14 may take the form of a throttle position 
15 sensor, and the steering column sensor 16 may take the form of a 
wheel position sensor. 

The sensors can also include a speed input sensor 23 
including a toothed wheel that rotates as the vehicle moves and 
produces pulsed signals in accordance with the rotational speed of 
the wheel . Another optional sensor is a Global Positioning 
Satellite receiver 52 which receives positional signals through 
its roof -mounted antenna 54 and produces sensor signals that are 
used by the sensor data recorder 10 to indicate the global 
location of the vehicle in which it is installed (within 30 
meters) . A further optional sensor is a door open sensor 56. The 
door open sensor 56 can be useful in identifying situations in 
which it is desirable to start recording data, even if the vehicle 
is not started. It can also be useful in identifying situations in 
which a public safety officer has left the vehicle, signifying 
that a beacon- following system controlling the pointing direction 
of the CCD microcamera(s) 22 and 25 should be activated. 
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The mass storage device 30 of the recorder 24 can be a 
removable hard disk having 500 megabytes of storage capacity. The 
recording time for the mass storage device is dependent upon the 
sampling rate of the signals which are to be stored. 
5 The signal conversion circuit 36 is connected to one or both 

of the CCD microcameras 22 and 25 and includes flash converter 
circuitry to quickly transform the analog video signals from the 
microcameras to digital form. In this form, and in coordination 
with the signals produced by a system timing circuit, a frame 

10 buffer circuit produces framed digitized video data that are 

i 

transferred to the bus 32 for transport to other hardware 
components of the data recorder 10, such as the signal processing 
circuit 34. 

The signal processing circuit 34 includes a CPU, such as an 

is Intel 80386 microprocessor, which is connected to a dynamic RAM 
and to a real-time clock in accordance with the good engineering 
practices well-known to those skilled in the electronics hardware 
arts. The data produced by the microprocessor are transmitted to 
the bus 32, which transfers them to the appropriate hardware 

20 component of the sensor data recorder 10 # such as the mass storage 
device 30. In specific application, the microprocessor of the 
signal processing circuit 34 receives the data containing header, 
video and sensor data according to the file format 60 (see Figure 
2) . As the data are received, the microprocessor applies a 

25 compression algorithm, such as the LWZ algorithm, to the received 
data and stores the data in compressed form in the dynamic RAM. 
The microprocessor then applies an encryption algorithm to the 
stored data and transfers the resulting encrypted and compressed 
data to the mass storage device 30 through the bus 32 . In the mass 

30 storage device 30, the data are stored in a hard disk that is 

removable from the mass storage device 30. The removable hard disk 
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containing the encrypted compressed data can be stored in a 
convenient location for later retrieval and replay of the data. 

The power supply 70 of the data recorder 10 shown in Figure 3 
is an integral part of the data recorder. The power supply 70 is 
5 normally connected to the vehicle* s battery. However, to protect 
the data recorder 10 from unexpected disruption and/or intended 
sabotage to the connection between the power supply 70 and the 
battery, the power supply 70 includes a backup battery 72. The 
backup battery f 70 will supply power to the data recorder 10 in the 
10 event that the connection between the power supply 70 and the 

vehicle's battery is broken. In addition, the power supply 70 can 
be used to extend the time that the data recorder 10 can record 
data for a period of time after the automobile's ignition switch 
has been turned off. 
15 Figure 4 is a flow chart of the program running within the 

preferred embodiment of the invention shown in Figure 1 . The 
program is simultaneously accomplishing two tasks. One task is to 
receive and process data from the sensors. The other task is to 
receive and process data from the video camera (s) 22 (25) and to 
20 combine the video data with the sensor data. The combined data is 
then compressed, encrypted and stored to disk for later retrieval 
and replay in accordance with methods known in the prior art. The 
replay methods comprise the inverse steps to those described below 
in carrying out the encoding, encryption, compression and r 
25 recording steps. 

In accordance with signals produced by the system clock 
(block 100), the video data task starts (block 102) and digitizes 
a frame of video data produced by one or both of the microcameras 
(block 104) . The system clock signals control when the readings 
30 from the sensors and the video camera (s) are taken. Typically, the 
sensor signals are sampled four times per second and the video 
camera data are sampled once per minute. However, the data can be 
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sampled at different controllable rates as described below. The 
digitized video data is transferred to the dynamic RAM (block 106) 
and interspersed with time/date signals produced by the system 
clock, the time/date signals characterizing the time and date at 
5 which the signals are produced (block 108) . The sensor data are 

also appended to the stored video and time/date signal data (block 
110) in the file format described above, compressed (block 112) , 
encrypted (block 114), and stored to disk (block 116) for later 
retrieval and replay. Then this task of the computer program 
10 awaits a further signal from the system clock, at which time it 
will begin again at block 102 . 

In the other task, the program reads the sensors when the 
appropriate signal is produced by the system clock (block 118) . 
The sensor readings are taken frequently, allowing them to be 
is processed digitally and/or statistically before they are made 
available as sensor data by the step represented by block 118. 
This program task then compares the sensor data to trip points or 
thresholds (block 120) and determines whether an alarm should be 
produced (block 122) . In the alarm condition, when the CCD 
20 microcameras are normally inactive, they are activated to record 

the events that caused, and the sensor data that resulted from the 
alarm condition. However, if the CCD video microcameras are 
normally active, their signals may be sampled more frequently in 
the case of an alarm to increase the chance that the event 
25 prompting the alarm condition will be completely recorded. 

Examples of such trip points are a particular vehicle speed or a 
particular amount of steering wheel motion. In the first example, 
the step of block 120 can detect when the vehicle speed exceeds 
the trip point, and the step of block 122 can determine that an 
30 alarm should be produced. In the second example, the data recorder 
analyzes the magnitude and frequency of the steering motions to 
establish when the trip point has been exceeded. 
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If an alarm is to be produced, the alarm signal is processed 
(block 124), causing adjustment in the sampling rate, an alarm 
timeout and reset condition are established (block 126) and the 
program returns to block 118 to read the sensors again. To save 
5 storage space in the mass storage device 30, the alarm timeout is 
established to allow the system to collect possibly relevant data 
without requiring an undesirably large amount of the recording 
medium. The duration of the alarm- timeout is a function of the 
reason for the ^ alarm. For example, if the alarm is caused by the 
10 vehicle's speed exceeding a desired threshold, the data recorder 
10 should be activated only as long as the speed is above the 
desired threshold. On the other hand, if the alarm is caused by 
erratic steering movements, the alarm condition should last beyond 
the time when the erratic steering movements cease. 
15 If the alarm is not to be produced, the sensor data are 

stored in an array in memory (block 128) for use by the append 
sensor data step (represented by block 110) of the other task of 
the program, and the present task of the program returns to block 
118 to read the sensors again. 
20 Figure 5 is a representation of a video image, illustrating a 

method for controlling and assuring the integrity of the image 
upon processing, storage and playback. In applications where 
compression of the entire video image would be impossible and yet 
the integrity of the video image 80 must be preserved (such as in 
is a real-time recording and playback system) , it is possible to 
sample and compress only a portion of the video signal from the 
CCD microcamera 22 and then to combine this signal with the 
signals from the various sensors 11, 12, 14, 16, 18, 20 and 21 
before encrypting the resulting data. The video image 80 can 
o include an image area 82 and generally an area 84 in which the 
date, time and status of various sensors (SI, S2 and S3) are 
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displayed. In addition, both the entire video image 80 and a 
subportion 86 of the video image 80 are compressed. 

The entire video image 80 need not be encrypted before 
storage, since the subportion 86 of the video image 80 is 
encrypted and preserved for decryption and comparison to the 
stored image 70 to assure that the stored image is valid. Instead 
the entire video image 80 is converted to the file format but not 
combined with any sensor data. Rather, the subportion 86 is 
compressed and combined with the sensor data and then encrypted. 
Parameters giving the location of the subportion 86 are 
incorporated into the header data so that it will be possible to 
correctly compare the decrypted and decompressed subportion 80 
with the appropriate part of the decompressed entire video image 
80. The subportion 86 can be placed at any desired location in the 
entire video image 80. However, if desired, the location of the 
subportion can be changed randomly (say, to a subportion 86') so 
that a potential counterfeiter would be unable to know which 
subportion 86, if any, of the entire image 80 was used to validate 
the played-back compressed entire video image 80 . 

Figure 6 is a block diagram of an apparatus for playback of 
the data stored by the data recorder of the present invention. The 
playback apparatus 140 includes a playback unit 142, a decryption 
circuit 144, a decompression circuit 146, a playback signal 
processing circuit 148, a video display unit (VDU) 150, a speaker 
152 and a sensor display 154. The playback unit # 142 includes a 
control panel 155 from which a user of the playback apparatus 140 
can control the playback process, including entry of any data 
needed to facilitate the decryption process, such as entry of the 
second (unpublished) component of any public key encryption system 
that may be in use. The playback unit 142 accepts the removable 
hard disk that was removed from the mass storage device 30, and 
plays back the removable hard disk to reproduce the digital data 
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stored on the removable hard disk. The signals representing the 
digital data are transmitted to the decryption circuit 144 which 
removes the effect of the encryption performed on the data prior 
to storage on the removable hard disk drive. The decryption 
circuit 144 produces a compressed signal and an uncompressed 
signal, including original sensor signals. The compressed signal 
is received by the decompression circuit 14 6 which produces a 
decompressed video signal and a decompressed sensor signal . The 
uncompressed signal from the decryption circuit 144 and both of 
the decompressed signals from the decompression circuit 146 are 
received by the signal processing circuit 148. In response, the 
signal processing circuit 148 produces an analog video signal and 
an audio signal . The analog video signal is received by the VDU 
150 which causes the VDU 150 to reproduce the original images. 
Also, the speaker 152 receives the audio signal and reproduces the 
original audio signals that accompanied the original images. In 
addition, the sensor display 154 receives the sensor signals and 
reproduces their original readings. 

Figure 7 is a flow chart of the method of playing back the 
data stored by the data recorder of the present invention. In this 
method, the digital signals recorded on the removable hard disk 
are played back (block 160) . Next, the encrypted digital signals 
among the played-back signals are decrypted (block 162) . Any 
compressed signals included among those that were originally 
played back or later decrypted are then decompressed (block 164) 
and the resulting uncompressed and decompressed signals are 
processed to produce audio signals, video signals and sensor 
signals that were originally concurrent (block 166). The audio 
signals are then played, and the concurrent video and sensor 
signals are displayed (block 168), thereby reproducing the 
original video, audio and sensor signals. In the case where only a 
portion of the original video image is encrypted, the signal 
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processing circuit 148 also compares the decompressed decrypted 
subportion 86 of the entire video image 82 with the proper portion 
of the decompressed entire video image 82 before displaying the 
decompressed entire video image 82. If there are discrepancies, 
5 the signal processing circuit 148 will alert the user to the type 
and location of the discrepancies. 

The preferred embodiment of the present invention has been 
described in detail sufficient for one skilled in the electronics 
o and radio frequency arts to understand the invention. Such skilled 
persons, however, could devise alternative embodiments to that 
described herein while remaining within the scope of the appended 
claims. Accordingly, the scope of the invention is to be limited 
only by the appended claims. 
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Claims 
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1. A method for recording sensor data with a resultant 
video signal to protect the sensor data and the resultant video 
signal, the method comprising the steps of: 

a) selecting one or more sensors that produce the sensor 

data,* 

b) transforming the signals from the one or more sensors 
to digital signals, including digitizing analog signals produced 
by some of the sensors and combining the digitized analog signals 
with digital signals produced by the remaining sensors to produce 
a digital sensor signal; 

c) selecting one or more video signal sources from a set 
of two or more video signal sources producing respective video 
signals; 

d) producing the resultant video signal from the 
respective video signals produced by the selected one or more 
video signal sources; 

20 e) encrypting at least one of the digital sensor signal 

and the resultant video signal in accordance with a digital 
encryption procedure to produce a respective digital output sensor 
signal and a respective digital output video signal ,- 

f) combining the respective digital output sensor signal 
25 and the respective digital output video signal to produce a 

digital output signal; and 

g) recording the digital output signal on a recording 

medium . 

30 2 - The method of claim 1, wherein step e) includes 

encrypting only the digital sensor signal to produce a digital 
output sensor signal . 
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3. The method of claim 1, wherein step e) includes 
encrypting only the digital video signal to produce a digital 
output video signal . 

5 

4. The method of claim 1, wherein step c) includes 
selecting the video signal sources as a function of conditions. 

5. The method of claim 4, wherein the conditions include 
10 the values of the data produced by one or more of the sensors. 

6. The method of claim 4, wherein the conditions include 
the values of the data produced by a sensor that measures a 
velocity . 

15 

7. The method of claim 4, wherein the conditions include 
the values of the data produced by a sensor that produces data 
representative of position. 

20 8. The method of claim 4, wherein the conditions include 

the values of the data produced by a sensor that is actuable by a 
human. 

9. The method of claim 4, wherein the conditions include 

2 5 the values of the data produced by a sensor that measures an 

i 

acceleration. 

10. The method of claim 4, wherein the conditions include 
the values of the data produced by a sensor that measures an 

30 acceleration. 
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11. The method of claim 10 , wherein the sensor that 
measures an acceleration measures accelerations in two different 
directions . 

5 12.. The method of claim 10, wherein the sensor that 

measures an acceleration measures accelerations in two different 
directions . 



10 



15 



13. The method of claim 1, wherein step c) includes 
causing the one or more video signal sources to move so that the 
video signal sources produce signals representative of changeable 
directions . 

14. The method of claim 1, wherein step a) includes 
selecting the sensor signal sources as a function of conditions. 

15. The method of claim 14, wherein the conditions include 
the values of the data produced by one or more of the sensors. 

16. The method of claim 14, wherein the conditions include 
the values of the data produced by a sensor that measures a 
velocity. 



17. The method of claim 14, wherein the conditions ; include 
25 the values of the data produced by a sensor that produces data 

representative of position. 

18. The method of claim 14, wherein the conditions include 
the values of the data produced by a sensor that is actuable by a 

3 0 human. 
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19. The method of claim 14, wherein the conditions include 
the values of the data produced by a sensor that measures an 
acceleration. 

5 20. The method of claim 14, wherein the conditions include 

the values of the data produced by a sensor that measures an 
acceleration. 

21. The method of claim 20, wherein the sensor that 

10 measures an acceleration measures accelerations in two different 
directions . 

22. The method of claim 20, wherein the sensor that 
measures an acceleration measures accelerations in two different 

is directions. 

23 . An apparatus for recording sensor data with a 
resultant video signal to protect the sensor data and the 
resultant video signal, the apparatus comprising: 
20 a selection circuit to select one or more sensors that 

produce the sensor data; 

a digitization circuit to transform the signals from the one 

or more sensors to digital signals, the digitization circuit 

digitizing analog signals produced by some of the sensors and 

25 combining the digitized analog signals with digital signals 

t 

produced by the remaining sensors to produce a digital sensor 
signal ,- 

a selection circuit to select one or more video signal 
sources from a set of two or more video signal sources producing 
30 respective video signals; 



-32- 



INSDOCID: <WO 9723075A1JA> 



WO 97/23075 



PCTYUS96/20912 



a video production circuit to produce the resultant video 
signal from the respective video signals produced by the selected 
one or more video signal sources,- 

an encryption circuit to encrypt at least one of the digital 
5 sensor signal and the resultant video signal in accordance with a 
digital encryption procedure and to produce a respective digital 
output sensor signal and a respective digital output video signal; 

a combination circuit to combine the respective digital 
output sensor ( signal and the respective digital output video 
10 signal to produce a digital output signal; and 

a recorder to record the digital output signal on a 
recording medium. 



24. The apparatus of claim 23, wherein the encryption 
is circuit encrypts only the digital sensor signal to produce a 

digital output sensor signal. 

25. The apparatus of claim 23, wherein the encryption 
circuit encrypts only the digital video signal to produce a 

20 digital output video signal. 

26. The apparatus of claim 23, wherein the selection 
circuit selects the video signal sources as a function of 



25 



conditions. 

27. The apparatus of claim 26, wherein the selection 
circuit receives values of the data produced by one or more of the 
sensors . 



30 28 * Th © apparatus of claim 26, wherein the conditions 

include the values of the data produced by a sensor that measures 
a velocity. 
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29. The apparatus of claim 26, wherein the conditions 
include the values of the data produced by a sensor that produces 
data representative of position. 

5 

30. The apparatus of claim 26, wherein the conditions 
include the values of the data produced by a sensor that is 
actuable by a human. 

10 31. The apparatus of claim 26, wherein the conditions 

include the values of the data produced by a sensor that measures 
an acceleration. 



32. The apparatus of claim 26, wherein the conditions 

15 include the values of the data produced by a sensor that measures 
an acceleration . 

33. The apparatus of claim 32, wherein the sensor that 
measures an acceleration measures accelerations in two different 

20 directions. 



34. The apparatus of claim 32, wherein the sensor that 
measures an acceleration measures accelerations in two different 
directions . 

25 

t 

35. The apparatus of claim 23, wherein step c) includes 
causing the one or more video signal sources to move so that the 
video signal sources produce signals representative of changeable 
directions . 
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36. The apparatus of claim 23, wherein the selection 
circuit selects the sensor signal sources as a function of 
conditions , 

5 37. The apparatus of claim 36, wherein the conditions 

include the values of the data produced by one or more of the 
sensors . 

38. The apparatus of claim 36, wherein the conditions 

10 include the values of the data produced by a sensor that measures 
a velocity. 

39. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that produces 

15 data representative of position. 

40. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that is 



20 



actuable by a human. 

41. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that measures 
an acceleration. 

42. The apparatus of claim 36, wherein the conditions 
include the values of the data produced by a sensor that measures 
an acceleration. 



43. The apparatus of claim 42, wherein the sensor that 
30 measures an acceleration measures accelerations in two different 
directions . 



25 
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44. The apparatus of claim 42, wherein the sensor that 
measures an acceleration measures accelerations in two different 
directions . 



5 45 . An apparatus for recording sensor data with a 

resultant video signal to protect the sensor data and the 
resultant video signal, the apparatus comprising: 

means for selecting one or more sensors that produce the 
sensor data; 

io means for transforming the signals from the one or more 

sensors to digital signals, the means for transforming the signals 
digitizing analog signals produced by some of the sensors and 
combining the digitized analog signals with digital signals 
produced by the remaining sensors to produce a digital sensor 

is signal; 

means for selecting one or more video signal sources from a 
set of two or more video signal sources producing respective video 
signals ; 

means for producing the resultant video signal from the 
20 respective video signals produced by the selected one or more 
video signal sources; 

means for encrypting at least one of the digital sensor 
signal and the resultant video signal in accordance with a digital 
encryption procedure and producing a respective digital output 
25 sensor signal and a respective digital output video signal; 

means for combining the respective digital output sensor 
signal and the respective digital output video signal and 
producing a digital output signal therefrom; and 

means for recording the digital output signal on a recording 

3 0 medium. 
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46. The apparatus of claim 45, wherein the means for 
encrypting encrypts only the digital sensor signal to produce a 
digital output sensor signal. 

5 47. The apparatus of claim 45, wherein the means for 

encrypting encrypts only the digital video signal to produce a 
digital output video signal . 

48. The apparatus of claim 45, wherein the means for 
10 selecting selects the video signal sources as a function of 

conditions . 

49. The apparatus of claim 48, wherein the means for 
selecting receives values of the data produced by one or more of 

is the sensors. 

50. The apparatus of claim 48, wherein the conditions 
include the values of the data produced by a sensor that measures 
a velocity. 

20 

51. The apparatus of claim 48, wherein the conditions 
include the values of the data produced by a sensor that produces 
data representative of position. 

25 52 • The apparatus of claim 48, wherein the conditions 

include the values of the data produced by a sensor that is 
actuable by a human. 



30 



53. The apparatus of claim 48, wherein the conditions 
include the values of the data produced by a sensor that measures 
an acceleration. 
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54. The apparatus of claim 48, wherein the conditions 
include the values of the data produced by a sensor that measures 
an acceleration. 



5 55. The apparatus of claim 45, wherein the means for 

selecting selects the sensor signal sources as a function of 
conditions. 

56. A method for recording sensor data with a video signal 
xo to protect the sensor data, the sensor data being contained in 

signals from one or more sensors, the method comprising the steps 

Of : 

a) encrypting the signals from the one or more sensors 

by: 

15 al) transforming the signals from the one or more 

sensors to digital signals, including digitizing analog signals 
produced by some of the sensors and combining the digitized analog 
signals with digital signals produced by the remaining sensors; 
and 

20 a2) encrypting the digital signals in accordance with 

a digital encryption procedure ; 

b) combining the video signal and the encrypted signals 
to produce an output signal; and 

c) recording the output signal on a recording medium. 

25 

57 . A method for retrieving sensor data originally 
produced by one or more sensors, a video signal and two or more 
signals representing the sensor data being encrypted to produce a 
recorded signal that protects the sensor data, the method 
30 comprising the steps of: 

a) playing back the recorded signal to produce a played- 
back signal; 
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b) decrypting the played-back signal to produce a video 
signal and one or more sensor signals containing data originally 
produced by one or more sensors; and 

c) processing the one or more sensor signals to produce 
5 the sensor data. 



58 . A method for retrieving sensor data originally 
produced by one or more sensors, a video signal and two or more 
signals representing the sensor data being encrypted to produce a 
recorded signal that protects the sensor data, the method 
comprising the steps of: 



10 



a) playing back the recorded signal to produce a played - 
back signal ; 

15 b > decrypting the played-back signal to produce a video 

signals and one or more sensor signals containing data originally 
produced by one or more sensors ; 

c) determining whether any of the one or more sensor 
signals are digital signals; and 

20 d > processing the one or more sensor signals to produce 

the sensor data. 

59. The method of claim 58, wherein at least one of the 
sensor signals is a digital signal that has been encrypted with a 

25 digital encryption procedure. 

60. The method of claim 59, wherein the digital encryption 
procedure is a public key encryption procedure. 



30 



61. The method of claim 60, wherein step d) includes 
decompressing any digital sensor signals. 
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62. A method for recording sensor data with a video signal 
to protect the sensor data, the sensor data being contained in 
signals from one or more sensors, the method comprising the steps 

Of: 

5 a) determining which of the signals from the one or more 

sensors are analog sensor signals and which of the signals from 
the one or more sensors are digital sensor signals; 

b) converting the data in the analog sensor signals to 
one or more first digital signals; 
io c) encrypting the digital sensor signals and the one or 

more first digital signals to produce second digital signals; 

d) converting the second digital signals to first analog 
signals ; and 

e) recording the video signal and the first analog 
is signals on a recording medium. 

63. The method of claim 62, wherein step c) comprises: 
cl) encrypting the digital sensor signals in accordance 

with a digital encryption procedure. 

20 

64. The method of claim 63, wherein step cl) includes 
encrypting the digital signals in accordance with a public key 
encryption procedure . 

25 65. The method of claim 63, wherein step, d) comprises: 

dl) compressing the second digital signals in accordance 
with a digital compression procedure. 

66 . A method for retrieving sensor data originally 
30 produced by one or more sensors, the sensor data being encrypted 

with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 
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sensors, at least one of the sensors producing an analog signal, 
the method comprising the steps of : 

a) playing back the recorded signals to produce a played- 
back signal; 

5 b) decrypting the played-back signal to produce a video 

signals and one or more sensor signals containing data originally 
produced by one or more sensors; 

c) determining whether any of the one or more sensor 
signals are digital signals; and 
10 d) processing the one or more sensor signals to produce 

the sensor data, including converting the one or more sensor 
signals to produce a corresponding analog signal. 

67. The method of claim 66, wherein at least one of the 
15 sensor signals is a digital signal that has been encrypted with a 
digital encryption procedure, and step d) including decrypting the 
digital signal according to the digital encryption procedure. 



20 



25 



30 



68. The method of claim 67, wherein the digital encryption 
procedure is a public key encryption procedure. 

69. The method of claim 68, wherein step d) includes 
decompressing any digital sensor signals. 

70. Apparatus for recording sensor data with a video 
signal, the sensor data being contained in signals from one or 
more sensors, the apparatus comprising: 

an encryption circuit to encrypt the signals from the one or 
more sensors; 

a signal processing circuit to combine the video signal and 
the encrypted signals to produce an output signal; and 
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a recorder to record the output signal on a recording 

medium. 

71 . Apparatus for retrieving sensor data originally 

5 produced by one or more sensors, the sensor data being encrypted 
with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 
sensors, the method comprising the steps of: 

a play-back circuit to play back the recorded signal to 
10 produce a played-back signal; 

a decryption circuit to decrypt the played-back signal to 
produce a video signal and one or more sensor signals containing 
data originally produced by one or more sensors; and 

a signal processing circuit to process the one or more 
is sensor signals to produce the sensor data. 

72. The apparatus of claim 71, wherein at least one of the 
sensor signals is a digital signal that has been encrypted with a 
digital encryption procedure. 

20 

73. The apparatus of claim 71, wherein the digital 
encryption procedure is a public key encryption procedure. 

74. Apparatus for recording sensor data with a video 

25 signal to protect the sensor data, the sensor data being contained 
in signals from one or more sensors, the apparatus comprising: 

a compression circuit to compress the video signal 
including: 

a digitizer to digitize the video signal to produce 
3 0 video digital signals therefrom; and 

a digital compression circuit to compress the video 
digital signals in accordance with a digital compression procedure 
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and produce the compressed video signal, the digital compression 
circuit including an encryption circuit to encrypt the digital 
signals in accordance with a digital encryption procedure ; 

a signal processing circuit to combine the compressed video 
signal and the signals from the one or more sensors and to produce 
a digital signal therefrom; 

an encryption circuit to encrypt the digital signal and 
produce an output signal therefrom; and 

a recorder to record the output signal on a recording 

medium . 

75. The apparatus of claim 74, wherein the encryption 
circuit includes a public key encryption circuit to encrypt the 
digital signals in accordance with a public key encryption 

is procedure. 

76. The apparatus of claim 74, wherein the video signal 
and the signals containing the sensor data are digital signals, 
the compression circuit includes a digital compression circuit to 

20 compress the digital signals in accordance with a digital 

compression procedure, the signal processing circuit digitally 
combines the video signal and the encrypted signals, and the 
encryption circuit encrypts the digital signal in accordance with 
a digital encryption procedure. 

25 

77 . Apparatus for retrieving sensor data originally 
produced by one or more sensors, the sensor data being encrypted 
with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 

30 sensors, the apparatus comprising: 

a play-back circuit to play back the recorded signal to 
produce a played-back signal; 

-43- 



5 



10 



BNSDOC1D: <WO 9723075A1_IA ? 



WO 97/23075 



PCT/US96/20912 



a decryption circuit to decrypt the played-back signal to 
produce a video signals and one or more sensor signals containing 
data originally produced by one or more sensors ; 

a logic circuit to determine whether any of the one or more 
5 sensor signals are digital signals; and 

a signal processing circuit to process the one or more 
sensor signals to produce the sensor data. 

78. The apparatus of claim 77, wherein at least one of the 
io sensor signals is a digital signal that has been encrypted with a 

digital encryption procedure. 

79. The apparatus of claim 78, wherein the digital 
encryption procedure is a public key encryption procedure. 

15 

80. The method of claim 78, wherein the signals processing 
circuit includes a decompression circuit to decompress any digital 
sensor signals. 

81. Apparatus for recording sensor data with a video 
signal, the sensor data being contained in signals from one or 
more sensors, the apparatus comprising: 

a distinction circuit to determine which of the signals from 
the one or more sensors are analog sensor signals and which of the 
signals from the one or more sensors are digital sensor signals; 

a first conversion circuit to convert the data in the analog 
sensor signals to one or more first digital signals,* 

an encryption circuit to encrypt the digital sensor signals 
and the one or more first digital signals to produce second 
digital signals; 

a second conversion circuit to convert the second digital 
signals to first analog signals; and 
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15 



20 



25 
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a recorder to record the video signal and the first analog 
signals on a recording medium. 

82. The apparatus of claim 81, wherein the encryption 
circuit comprises: 

a digital encryption circuit to encrypt the digital sensor 
signals in accordance with a digital encryption procedure. 

83 . The apparatus of claim 82 , wherein the digital 
encryption circuit includes a public key encryption circuit to 
encrypt the digital signals in accordance with a public key 
encryption procedure . 

84. The apparatus of claim 81, wherein the second 
conversion circuit includes a compression circuit to compress the 
second digital signals in accordance with a digital compression 
procedure . 

85 . Apparatus for retrieving sensor data originally 
produced by one or more sensors, the sensor data being encrypted 
with a video signal to produce a recorded signal that protects the 
sensor data and being contained in signals from one or more 
sensors, at least one of the sensors producing an analog signal, 
the apparatus comprising: 

a play-back circuit to play back the recorded signal to 
produce a played-back signal; 

a decryption circuit to decrypt the played-back signal to 
produce a video signal and one or more sensor signals containing 
data originally produced by one or more sensors: 

a logic circuit to determine whether any of the on sensor 
signals are digital signals; and 
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a signal processing circuit to process the one or more 
sensor signals to produce the sensor data, including converting 
the one or more sensor signals to produce a corresponding analog 
signal . 

5 

86. The apparatus of claim 85, wherein at least one of the 
sensor signals is a digital signal that has been encrypted with a 
digital encryption procedure, and the signal processing circuit 
includes a decryption circuit to decrypt the digital signal 

10 accordion to the digital encryption procedure. 

87. The apparatus of claim 86, wherein the digital 
encryption procedure is a public key encryption procedure. 

is 88. The apparatus of claim 87, wherein the decryption 

circuit decompresses any digital sensor signals. 

89. Apparatus for recording sensor data with a video 
signal to protect the sensor data, the sensor data being contained 

20 in signals from one or more sensors, the apparatus comprising: 
means for encrypting the signals from the one or more 
sensors; 

means for combining the video signal and the encrypted 
signals to produce an output signal; and 
25 means for recording the output signal on ^ recording medium. 

90, Apparatus for recording sensor data with a video 
signal to protect the sensor data, the sensor data being contained 
in signals from one or more sensors, the apparatus comprising: 

30 means for compressing the video signal to produce a 

compressed video signal; 
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means for combining- the compressed video signal and the 
signals from the one or more sensors to produce a digital signal; 

means for encrypting the digital signal to produce an output 
signal; and 

5 means for recording the output signal on a recording medium. 

91. Apparatus for recording sensor data with a video 
signal to protect the sensor data, the sensor data being contained 
in signals from one or more sensors, the apparatus comprising: 

means for determining which of the signals from the one or 
more sensors are analog sensor signals and which of the signals 
from the one or more sensors are digital sensor signals ,- 

first means for converting the data in the analog sensor 
signals to one or more first digital signals,- 
15 means for encrypting the digital sensor signals and the one 

or more first digital signals to produce second digital signals; 

second means for converting the second digital signals to 
first analog signals; and 

means for recording the video signal and the first analog 
20 signals on a recording medium. 

92. A method for recording sensor data with a video 
signal, the sensor data being contained in signals from one or 
more sensors, the method comprising the steps of : 

25 a) encrypting the signals from the one or more sensors to 

produce an output signal; and 

b) recording the output signal and the video signal on a 
recording medium. 



30 



93. Apparatus for recording sensor data with a video 
signal, the sensor data being contained in signals from one or 
more sensors, comprising: 
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an encryption circuit to encrypt the signals from the one or 
more sensors to produce an output signal; and 

a recorder to record the output signal and the video signal 
on a recording medium. 

5 

94 . A method for recording sensor data with a video signal 
in a motor vehicle, the sensor data being contained in signals 
from one or more sensors in the motor vehicle, the method 
comprising the steps of : 
10 a) encrypting the signals from the one or more sensors ; 

b) combining the video signal and the encrypted signals 
to produce an output signal ; and 

c) recording the output signal on a recording medium. 

15 95. Apparatus for recording sensor data with a video 

signal in a motor vehicle, the sensor data being contained in 
signals from one or more sensors in the motor vehicle, the 
apparatus comprising : 

an encryption circuit to encrypt the signals from the one or 
20 more sensors 

a circuit to combine the video signal and the encrypted 
signals and produce an output signal therefrom; and 

a recorder to record the output signal on a recording 
medium. r 

25 
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